Code Copilot AI生成代码浅测
最编程
2024-07-31 13:32:53
...
引言
评测背景
近年来，人工智能技术在软件开发领域取得了显著的进展，其中包括AI生成代码的应用。AI生成代码是指利用机器学习和自然语言处理等技术，让计算机能够根据给定的问题或需求自动生成代码。
AI生成代码的概念引发了广泛的研究和探索。传统的编码过程需要开发人员具备深入的编程知识和经验，而AI生成代码则提供了一种更加便捷和高效的方式来创建代码。它可以自动化部分编码任务，减少人为的错误和繁琐的重复工作，同时提高开发效率和生产力。
评测目的
评测AI生成代码的目的是为了深入了解该技术在软件开发中的应用和潜力。通过评测，我们可以评估AI生成代码的优势、限制和适用范围。
评测概述
本次评测旨在对AI生成代码技术进行全面评估，探索其在软件开发领域的应用和潜力。我们将对不同的AI生成代码模型和方法进行测试，以评估其生成代码的质量、效率和创新性。
涉及到的工具：
- GitHub Copilot
- GPT3.5
- Bard
- Bing
评测方法
评测维度
-
代码质量评估：
- 我们将对AI生成的代码进行质量评估，包括代码的结构、逻辑、命名规范、注释和可读性等方面。通过比较生成代码与手动编写代码的质量差异，我们可以评估AI生成代码在实际项目中的可用性和实用性。
-
开发效率比较：
- 我们将比较使用AI生成代码和传统手动编写代码的开发效率。通过测量开发时间、工作量和代码重用程度等指标，我们可以评估AI生成代码在加速开发过程和减少开发工作量方面的能力。
-
创新性和多样性分析：
- 我们将关注AI生成代码的创新性和多样性。评估AI系统在生成代码时是否能提供不同的解决方案和代码风格，以促进创新和灵活性。
-
领域适应性测试：
- 我们将测试不同领域和复杂度的编码任务，评估AI生成代码技术在特定领域的适应性和性能表现。
评测场景
- 代码生成
- 生成函数
- 生成API调用
- 类型注释
- UI组件
- 套路代码
- 问题修复
- 类型错误
- 未定义的变量
- 未捕捉的异常
- 未实现的接口方法
- 代码重构
- 合理的变量和函数命名
- 消除重复代码
- 简化条件语句
- 潜在的性能问题
- UT生成
- 纯函数
- 包含UI操作
- 依赖全局变量
- 包含异步操作
- 文档编写
- 纯函数
- UI组件
- 技术方案
评测结果
代码生成
生成函数
GitHub Copilot
/// 用dart生成一个函数把一个数组分成N个数组
List<List<int>> splitList(List<int> list, int n) {
var result = <List<int>>[];
var temp = <int>[];
for (var i = 0; i < list.length; i++) {
temp.add(list[i]);
if ((i + 1) % n == 0) {
result.add(temp);
temp = <int>[];
}
}
if (temp.isNotEmpty) {
result.add(temp);
}
return result;
}
GPT3.5
/// 用dart生成一个函数把一个数组分成N个数组
List<List<T>> splitArray<T>(List<T> array, int N) {
final int totalLength = array.length;
final int chunkSize = (totalLength / N).ceil();
final List<List<T>> result = [];
for (int i = 0; i < totalLength; i += chunkSize) {
final List<T> chunk = array.sublist(i, i + chunkSize);
result.add(chunk);
}
return result;
}
Bard
/// Use dart to generate a function to divide an array into N arrays
List<List<int>> divideArray(List<int> array, int n) {
if (n <= 0) {
throw ArgumentError('n must be positive');
}
List<List<int>> chunks = [];
for (int i = 0; i < array.length; i += n) {
chunks.add(array.sublist(i, min(i + n, array.length)));
}
return chunks;
}
Bing
/// 用dart生成一个函数把一个数组分成N个数组
List<List<T>> splitArray<T>(List<T> array, int n) {
if (n <= 0) throw ArgumentError('n must be positive');
final result = <List<T>>[];
final chunkSize = (array.length / n).ceil();
for (var i = 0; i < array.length; i += chunkSize) {
result.add(array.sublist(i, i + chunkSize));
}
return result;
}
生成API调用
GitHub Copilot
// 使用dio并用生成的rest发起一个POST请求，请求payload参数为 { name: '111',age: 333, } 请求path是 https://api.example.com/api/v1/test/ 把返回的数据打印出来
void postRequest() async {
// 生成的rest
const rest = 'https://api.example.com/api/v1/test/';
// 使用dio发起一个POST请求
// 参数 [rest]: 请求的地址
// 参数 [data]: 请求的数据
final response = await dio.post(rest, data: {
'name': '111',
'age': 333,
});
// 打印返回的数据
print(response.data);
}
GPT3.5
// 使用dio并用生成的rest发起一个POST请求，请求payload参数为 { name: '111',age: 333, } 请求path是 https://api.example.com/api/v1/test/ 把返回的数据打印出来
import 'package:dio/dio.dart';
void main() async {
final dio = Dio();
final payload = {
'name': '111',
'age': 333,
};
final url = 'https://api.example.com/api/v1/test/';
try {
final response = await dio.post(url, data: payload);
print(response.data);
} catch (e) {
print('Error: $e');
}
}
Bard
/// Use dio and use the generated rest to initiate a POST request, the request payload parameter is { name: '111', age: 333, } The request path is https://api.example.com/api/v1/test/ Send the returned data print it out
import 'package:dio/dio.dart';
void main() async {
// Create a Dio instance
Dio dio = Dio();
// Set the request headers
dio.options.headers['Content-Type'] = 'application/json';
// Create the request payload
Map<String, dynamic> data = {
'name': '111',
'age': 333,
};
// Initiate the POST request
Response response = await dio.post('https://api.example.com/api/v1/test/', data: data);
// Check the response status code
if (response.statusCode == 200) {
// Print the returned data
print(response.data);
} else {
// Handle the error
print('Error: ${response.statusCode}');
}
}
Bing
/// 使用dio并用生成的rest发起一个POST请求，请求payload参数为 { name: '111',age: 333, } 请求path是 https://api.example.com/api/v1/test/ 把返回的数据打印出来
import 'package:dio/dio.dart';
void main() async {
final dio = Dio();
final payload = {'name': '111', 'age': 333};
final response = await dio.post(
'https://api.example.com/api/v1/test/',
data: payload,
);
print(response.data);
}
类型注释
// 给下面的dart函数添加注释
String timePeriodCompositor(Map<String, dynamic> timePeriod) {
final bool hasMonday = timePeriod['hasMonday'];
final bool hasTuesday = timePeriod['hasTuesday'];
final bool hasWednesday = timePeriod['hasWednesday'];
final bool hasThursday = timePeriod['hasThursday'];
final bool hasFriday = timePeriod['hasFriday'];
final bool hasSaturday = timePeriod['hasSaturday'];
final bool hasSunday = timePeriod['hasSunday'];
final bool hasLegalHolidays = timePeriod['hasLegalHolidays'];
final int startHour = timePeriod['startHour'];
final int startMinute = timePeriod['startMinute'];
final int endHour = timePeriod['endHour'];
final int endMinute = timePeriod['endMinute'];
final Map<int, String> dayMap = {
1: '周一',
2: '周二',
3: '周三',
4: '周四',
5: '周五',
6: '周六',
7: '周日',
};
final List<int> dayArray = [
hasMonday ? 1 : 0,
hasTuesday ? 2 : 0,
hasWednesday ? 3 : 0,
hasThursday ? 4 : 0,
hasFriday ? 5 : 0,
hasSaturday ? 6 : 0,
hasSunday ? 7 : 0,
];
final String dayDesc = dayArray
.join('')
.split('0')
.map((chunk) {
switch (chunk.length) {
case 0:
return '';
case 1:
return dayMap[int.parse(chunk[0])];
case 2:
return '${dayMap[int.parse(chunk[0])]}、${dayMap[int.parse(chunk[1])]}';
case 7:
return '每天';
default:
return '${dayMap[int.parse(chunk[0])]} ~ ${dayMap[int.parse(chunk[chunk.length - 1])]}}';
}
})
.where((str) => str?.isNotEmpty == true)
.join('、');
final String timeDesc =
'$startHour:$startMinute ~ ${endHour == 23 && endMinute == 59 ? '24:00' : '$endHour:$endMinute'}';
return '$dayDesc，$timeDesc，${hasLegalHolidays ? '' : '除'}法定节假日可用';
}
GitHub Copilot
// 给下面的dart函数添加类型注释
/// Composes a time period string based on the given map of time period data.
///
/// The time period data should contain the following keys:
/// - hasMonday: a boolean indicating whether the time period includes Mondays
/// - hasTuesday: a boolean indicating whether the time period includes Tuesdays
/// - hasWednesday: a boolean indicating whether the time period includes Wednesdays
/// - hasThursday: a boolean indicating whether the time period includes Thursdays
/// - hasFriday: a boolean indicating whether the time period includes Fridays
/// - hasSaturday: a boolean indicating whether the time period includes Saturdays
/// - hasSunday: a boolean indicating whether the time period includes Sundays
/// - hasLegalHolidays: a boolean indicating whether the time period includes legal holidays
/// - startHour: an integer indicating the starting hour of the time period
/// - startMinute: an integer indicating the starting minute of the time period
/// - endHour: an integer indicating the ending hour of the time period
/// - endMinute: an integer indicating the ending minute of the time period
///
/// Returns a string describing the time period in the format:
/// "{dayDesc}，{timeDesc}，{availabilityDesc}"
///
/// where:
/// - dayDesc: a string describing the days of the week the time period is available
/// - timeDesc: a string describing the time range of the time period
/// - availabilityDesc: a string indicating whether the time period is available during legal holidays
String timePeriodCompositor(Map<String, dynamic> timePeriod) {
final bool hasMonday = timePeriod['hasMonday'];
final bool hasTuesday = timePeriod['hasTuesday'];
final bool hasWednesday = timePeriod['hasWednesday'];
final bool hasThursday = timePeriod['hasThursday'];
final bool hasFriday = timePeriod['hasFriday'];
final bool hasSaturday = timePeriod['hasSaturday'];
final bool hasSunday = timePeriod['hasSunday'];
final bool hasLegalHolidays = timePeriod['hasLegalHolidays'];
final int startHour = timePeriod['startHour'];
final int startMinute = timePeriod['startMinute'];
final int endHour = timePeriod['endHour'];
final int endMinute = timePeriod['endMinute'];
final Map<int, String> dayMap = {
1: '周一',
2: '周二',
3: '周三',
4: '周四',
5: '周五',
6: '周六',
7: '周日',
};
final List<int> dayArray = [
hasMonday ? 1 : 0,
hasTuesday ? 2 : 0,
hasWednesday ? 3 : 0,
hasThursday ? 4 : 0,
hasFriday ? 5 : 0,
hasSaturday ? 6 : 0,
hasSunday ? 7 : 0,
];
final String dayDesc = dayArray
.join('')
.split('0')
.map((chunk) {
switch (chunk.length) {
case 0:
return '';
case 1:
return dayMap[int.parse(chunk[0])];
case 2:
return '${dayMap[int.parse(chunk[0])]}、${dayMap[int.parse(chunk[1])]}';
case 7:
return '每天';
default:
return '${dayMap[int.parse(chunk[0])]} ~ ${dayMap[int.parse(chunk[chunk.length - 1])]}}';
}
})
.where((str) => str?.isNotEmpty == true)
.join('、');
final String timeDesc =
'$startHour:$startMinute ~ ${endHour == 23 && endMinute == 59 ? '24:00' : '$endHour:$endMinute'}';
return '$dayDesc，$timeDesc，${hasLegalHolidays ? '' : '除'}法定节假日可用';
}
GPT3.5
/// Composes a time period description based on the provided [timePeriod] data.
///
/// The [timePeriod] should be a map with the following keys:
/// - 'hasMonday': bool, indicating if Monday is included in the time period.
/// - 'hasTuesday': bool, indicating if Tuesday is included in the time period.
/// - 'hasWednesday': bool, indicating if Wednesday is included in the time period.
/// - 'hasThursday': bool, indicating if Thursday is included in the time period.
/// - 'hasFriday': bool, indicating if Friday is included in the time period.
/// - 'hasSaturday': bool, indicating if Saturday is included in the time period.
/// - 'hasSunday': bool, indicating if Sunday is included in the time period.
/// - 'hasLegalHolidays': bool, indicating if legal holidays are included in the time period.
/// - 'startHour': int, the starting hour of the time period.
/// - 'startMinute': int, the starting minute of the time period.
/// - 'endHour': int, the ending hour of the time period.
/// - 'endMinute': int, the ending minute of the time period.
///
/// Returns a composed time period description string.
String timePeriodCompositor(Map<String, dynamic> timePeriod) {
final bool hasMonday = timePeriod['hasMonday'];
final bool hasTuesday = timePeriod['hasTuesday'];
final bool hasWednesday = timePeriod['hasWednesday'];
final bool hasThursday = timePeriod['hasThursday'];
final bool hasFriday = timePeriod['hasFriday'];
final bool hasSaturday = timePeriod['hasSaturday'];
final bool hasSunday = timePeriod['hasSunday'];
final bool hasLegalHolidays = timePeriod['hasLegalHolidays'];
final int startHour = timePeriod['startHour'];
final int startMinute = timePeriod['startMinute'];
final int endHour = timePeriod['endHour'];
final int endMinute = timePeriod['endMinute'];
final Map<int, String> dayMap = {
1: '周一',
2: '周二',
3: '周三',
4: '周四',
5: '周五',
6: '周六',
7: '周日',
};
final List<int> dayArray = [
hasMonday ? 1 : 0,
hasTuesday ? 2 : 0,
hasWednesday ? 3 : 0,
hasThursday ? 4 : 0,
hasFriday ? 5 : 0,
hasSaturday ? 6 : 0,
hasSunday ? 7 : 0,
];
final String dayDesc = dayArray
.join('')
.split('0')
.map((chunk) {
switch (chunk.length) {
case 0:
return '';
case 1:
return dayMap[int.parse(chunk[0])];
case 2:
return '${dayMap[int.parse(chunk[0])]}、${dayMap[int.parse(chunk[1])]}';
case 7:
return '每天';
default:
return '${dayMap[int.parse(chunk[0])]} ~ ${dayMap[int.parse(chunk[chunk.length - 1])]}}';
}
})
.where((str) => str?.isNotEmpty == true)
.join('、');
final String timeDesc =
'$startHour:$startMinute ~ ${endHour == 23 && endMinute == 59 ? '24:00' : '$endHour:$endMinute'}';
return '$dayDesc，$timeDesc，${hasLegalHolidays ? '' : '除'}法定节假日可用';
}
Bard
// Add comments to the following dart functions
/// This function composes a string that represents the time period.
///
/// The time period is represented by a map with the following keys:
///
/// * `hasMonday`: Whether Monday is included in the time period.
/// * `hasTuesday`: Whether Tuesday is included in the time period.
/// * `hasWednesday`: Whether Wednesday is included in the time period.
/// * `hasThursday`: Whether Thursday is included in the time period.
/// * `hasFriday`: Whether Friday is included in the time period.
/// * `hasSaturday`: Whether Saturday is included in the time period.
/// * `hasSunday`: Whether Sunday is included in the time period.
/// * `hasLegalHolidays`: Whether the time period includes legal holidays.
/// * `startHour`: The start hour of the time period.
/// * `startMinute`: The start minute of the time period.
/// * `endHour`: The end hour of the time period.
/// * `endMinute`: The end minute of the time period.
///
/// The function returns a string that has the following format:
///
/// `周一、周二 ~ 周五，10:00 ~ 12:00，法定节假日可用`
///
/// The string represents the time period as a list of days, start and end time, and whether legal holidays are included.
///
/// For example, the string `周一、周二 ~ 周五，10:00 ~ 12:00，法定节假日可用` represents a time period that includes Monday, Tuesday, Wednesday, Thursday, from 10:00 to 12:00, and is also available on legal holidays.
String timePeriodCompositor(Map<String, dynamic> timePeriod) {
final bool hasMonday = timePeriod['hasMonday'];
final bool hasTuesday = timePeriod['hasTuesday'];
final bool hasWednesday = timePeriod['hasWednesday'];
final bool hasThursday = timePeriod['hasThursday'];
final bool hasFriday = timePeriod['hasFriday'];
final bool hasSaturday = timePeriod['hasSaturday'];
final bool hasSunday = timePeriod['hasSunday'];
final bool hasLegalHolidays = timePeriod['hasLegalHolidays'];
final int startHour = timePeriod['startHour'];
final int startMinute = timePeriod['startMinute'];
final int endHour = timePeriod['endHour'];
final int endMinute = timePeriod['endMinute'];
final Map<int, String> dayMap = {
1: '周一',
2: '周二',
3: '周三',
4: '周四',
5: '周五',
6: '周六',
7: '周日',
};
final List<int> dayArray = [
hasMonday ? 1 : 0,
hasTuesday ? 2 : 0,
hasWednesday ? 3 : 0,
hasThursday ? 4 : 0,
hasFriday ? 5 : 0,
hasSaturday ? 6 : 0,
hasSunday ? 7 : 0,
];
final String dayDesc = dayArray
.join('')
.split('0')
.map((chunk) {
switch (chunk.length) {
case 0:
return '';
case 1:
return dayMap[int.parse(chunk[0])];
case 2:
return '${dayMap[int.parse(chunk[0])]}、${dayMap[int.parse(chunk[1])]}';
case 7:
return '每天';
default:
return '${dayMap[int.parse(chunk[0])]} ~ ${dayMap[int.parse(chunk[chunk.length - 1])]}}';
}
})
.where((str) => str?.isNotEmpty == true)
.join('、');
final String timeDesc =
'$startHour:$startMinute ~ ${endHour == 23 && endMinute == 59 ? '24:00' : '$endHour:$endMinute'}';
return '$dayDesc，$timeDesc，${hasLegalHolidays ? '' : '除'}法定节假日可用';
}
Bing
/// 组合时间段描述
///
/// [timePeriod] 是一个包含时间段信息的Map，它应该包含以下键值对：
/// - 'hasMonday'：bool类型，表示是否包含周一
/// - 'hasTuesday'：bool类型，表示是否包含周二
/// - 'hasWednesday'：bool类型，表示是否包含周三
/// - 'hasThursday'：bool类型，表示是否包含周四
/// - 'hasFriday'：bool类型，表示是否包含周五
/// - 'hasSaturday'：bool类型，表示是否包含周六
/// - 'hasSunday'：bool类型，表示是否包含周日
/// - 'hasLegalHolidays'：bool类型，表示是否包含法定节假日
/// - 'startHour'：int类型，表示开始时间的小时数
/// - 'startMinute'：int类型，表示开始时间的分钟数
/// - 'endHour'：int类型，表示结束时间的小时数
/// - 'endMinute'：int类型，表示结束时间的分钟数
///
/// 返回一个字符串，描述给定的时间段。
String timePeriodCompositor(Map<String, dynamic> timePeriod) {
// 从timePeriod中提取各个值
final bool hasMonday = timePeriod['hasMonday'];
final bool hasTuesday = timePeriod['hasTuesday'];
final bool hasWednesday = timePeriod['hasWednesday'];
final bool hasThursday = timePeriod['hasThursday'];
final bool hasFriday = timePeriod['hasFriday'];
final bool hasSaturday = timePeriod['hasSaturday'];
final bool hasSunday = timePeriod['hasSunday'];
final bool hasLegalHolidays = timePeriod['hasLegalHolidays'];
final int startHour = timePeriod['startHour'];
final int startMinute = timePeriod['startMinute'];
final int endHour = timePeriod['endHour'];
final int endMinute = timePeriod['endMinute'];
// 定义一个星期几的映射表
final Map<int, String> dayMap = {
1: '周一',
2: '周二',
3: '周三',
4: '周四',
5: '周五',
6: '周六',
7: '周日',
};
// 根据timePeriod中的值创建一个星期几的数组
final List<int> dayArray = [
hasMonday ? 1 : 0,
hasTuesday ? 2 : 0,
hasWednesday ? 3 : 0,
hasThursday ? 4 : 0,
hasFriday ? 5 : 0,
hasSaturday ? 6 : 0,
hasSunday ? 7 : 0,
];
// 将dayArray转换为一个字符串，然后根据0来分割它
// 对于每个分割后的字符串，根据它的长度来生成描述
final String dayDesc = dayArray
.join('')
.split('0')
.map((chunk) {
switch (chunk.length) {
case 0:
return '';
case 1:
return dayMap[int.parse(chunk[0])];
case 2:
return '${dayMap[int.parse(chunk[0])]}、${dayMap[int.parse(chunk[1])]}';
case 7:
return '每天';
default:
return '${dayMap[int.parse(chunk[0])]} ~ ${dayMap[int.parse(chunk[chunk.length - 1])]}}';
}
})
.where((str) => str?.isNotEmpty == true)
.join('、');
// 组合开始和结束时间的描述
final String timeDesc =
'$startHour:$startMinute ~ ${endHour == 23 && endMinute == 59 ? '24:00' : '$endHour:$endMinute'}';
// 返回最终的描述
return '$dayDesc，$timeDesc，${hasLegalHolidays ? '' : '除'}法定节假日可用';
}
UI组件
GitHub Copilot
/// 用dart实现一个登录界面，有两种模式，验证码登录和密码登录 验证码登录时有两个输入项，手机号和验证码，输入的手机号合法后才可发送验证码，60秒内只可以发送一次验证码
import 'dart:async';
import 'package:flutter/material.dart';
class LoginScreen extends StatefulWidget {
@override
_LoginScreenState createState() => _LoginScreenState();
}
class _LoginScreenState extends State<LoginScreen> {
final _formKey = GlobalKey<FormState>();
final _phoneController = TextEditingController();
final _codeController = TextEditingController();
bool _isPasswordLogin = true;
bool _isSendingCode = false;
bool _canSendCode = true;
Timer? _timer;
int _countdown = 60;
@override
void dispose() {
_timer?.cancel();
_phoneController.dispose();
_codeController.dispose();
super.dispose();
}
void _startTimer() {
_timer = Timer.periodic(Duration(seconds: 1), (timer) {
setState(() {
_countdown--;
});
if (_countdown == 0) {
_timer?.cancel();
setState(() {
_canSendCode = true;
});
}
});
}
void _sendCode() {
setState(() {
_isSendingCode = true;
_canSendCode = false;
_countdown = 60;
});
_startTimer();
}
void _submit() {
if (_formKey.currentState!.validate()) {
// TODO: Perform login
}
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text('Login'),
),
body: Padding(
padding: const EdgeInsets.all(16.0),
child: Form(
key: _formKey,
child: Column(
crossAxisAlignment: CrossAxisAlignment.stretch,
children: [
SizedBox(height: 16.0),
Row(
children: [
Expanded(
child: ElevatedButton(
onPressed: () {
setState(() {
_isPasswordLogin = true;
});
},
child: Text('Password Login'),
style: ElevatedButton.styleFrom(
primary: _isPasswordLogin ? Colors.blue : null,
),
),
),
SizedBox(width: 16.0),
Expanded(
child: ElevatedButton(
onPressed: () {
setState(() {
_isPasswordLogin = false;
});
},
child: Text('Verification Code Login'),
style: ElevatedButton.styleFrom(
primary: !_isPasswordLogin ? Colors.blue : null,
),
),
),
],
),
SizedBox(height: 16.0),
TextFormField(
controller: _phoneController,
keyboardType: TextInputType.phone,
decoration: InputDecoration(
labelText: 'Phone Number',
),
validator: (value) {
if (value == null || value.isEmpty) {
return 'Please enter your phone number';
}
// TODO: Add phone number validation
return null;
},
),
SizedBox(height: 16.0),
if (_isPasswordLogin)
TextFormField(
obscureText: true,
decoration: InputDecoration(
labelText: 'Password',
),
validator: (value) {
if (value == null || value.isEmpty) {
return 'Please enter your password';
}
// TODO: Add password validation
return null;
},
),
if (!_isPasswordLogin)
Row(
children: [
Expanded(
child: TextFormField(
controller: _codeController,
keyboardType: TextInputType.number,
decoration: InputDecoration(
labelText: 'Verification Code',
),
validator: (value) {
if (value == null || value.isEmpty) {