|
4 | 4 | 🤗 <a href="https://huggingface.co/microsoft">Hugging Face</a>   |    📑 <a href="https://arxiv.org/#">Paper</a> |
5 | 5 | </p> |
6 | 6 |
|
7 | | -## Assets |
| 7 | +## Assets (this is temporary section and will be removed) |
8 | 8 | - Only new synthetic dataset |
9 | 9 | - Models and usage |
10 | 10 | - Training reciepe |
11 | 11 |
|
| 12 | +## Model Usage |
| 13 | +```python |
| 14 | +from transformers import AutoModelForCausalLM, AutoTokenizer |
| 15 | + |
| 16 | +model_name = "Microsoft/NextCoder-7B" |
| 17 | + |
| 18 | +model = AutoModelForCausalLM.from_pretrained( |
| 19 | + model_name, |
| 20 | + torch_dtype="auto", |
| 21 | + device_map="auto" |
| 22 | +) |
| 23 | +tokenizer = AutoTokenizer.from_pretrained(model_name) |
| 24 | + |
| 25 | +prompt = """ |
| 26 | +Fix the following function that divides two numbers to handle all the edge cases: |
| 27 | +
|
| 28 | +def divide(a, b) |
| 29 | + returm a/b |
| 30 | +""" |
| 31 | +messages = [ |
| 32 | + {"role": "user", "content": prompt} |
| 33 | +] |
| 34 | +text = tokenizer.apply_chat_template( |
| 35 | + messages, |
| 36 | + tokenize=False, |
| 37 | + add_generation_prompt=True |
| 38 | +) |
| 39 | +model_inputs = tokenizer([text], return_tensors="pt").to(model.device) |
| 40 | + |
| 41 | +generated_ids = model.generate( |
| 42 | + **model_inputs, |
| 43 | + max_new_tokens=1024 |
| 44 | +) |
| 45 | +generated_ids = [ |
| 46 | + output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) |
| 47 | +] |
| 48 | + |
| 49 | +response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] |
| 50 | +``` |
| 51 | + |
| 52 | +## Evaluation and Performanc |
| 53 | + |
| 54 | +| Models | HUMANEVALEDIT | CANITEDIT | AIDER | POLYGLOT | |
| 55 | +|--------|---------------|-----------|-------|----------| |
| 56 | +| QwenCoder-2.5-3B | 73.2 | 37.1 | 36.8 | - | |
| 57 | +| QwenCoder-2.5-3B-LoRA | 64.6 | 36.2 | 35.8 | - | |
| 58 | +| QwenCoder-2.5-3B-SFT | 76.2 | 32.4 | 30.1 | - | |
| 59 | +| **NextCoder-3B** | 75.6 | 42.4 | 37.6 | - | |
| 60 | +| QwenCoder-2.5-14B | 87.8 | 58.1 | 66.9 | 9.3 | |
| 61 | +| QwenCoder-2.5-14B-LoRA | 78.0 | 50.9 | 66.2 | 5.3 | |
| 62 | +| QwenCoder-2.5-14B-SFT | 79.9 | 42.4 | 36.8 | 3.1 | |
| 63 | +| **NextCoder-14B** | 89.8 | 60.2 | 72.2 | 12.2 | |
| 64 | +| QwenCoder-2.5-32B | **90.2** | 61.0 | 72.9 | 16.4 | |
| 65 | +| QwenCoder-2.5-32B-LoRA | 82.3 | 52.4 | 60.2 | 6.7 | |
| 66 | +| QwenCoder-2.5-32B-SFT | 81.7 | 49.5 | 66.9 | 8.4 | |
| 67 | +| **NextCoder-32B** | 88.9 | **62.4** | **74.7** | **21.9** | |
| 68 | + |
| 69 | +*Comparison of base QwenCoder-2.5 models of different sizes and their SELEKT-enhanced versions across three code editing benchmarks.* |
| 70 | + |
12 | 71 | ## Contributing |
13 | 72 |
|
14 | 73 | This project welcomes contributions and suggestions. Most contributions require you to agree to a |
|
0 commit comments