From 746536b6330f829aabc5f91725b33b5a75f86984 Mon Sep 17 00:00:00 2001
From: Rishabh Srivastava <rishabhsriv@gmail.com>
Date: Wed, 7 Feb 2024 15:13:30 +0800
Subject: [PATCH] Update README.md

---
 README.md | 22 ++++++++++++----------
 1 file changed, 12 insertions(+), 10 deletions(-)

diff --git a/README.md b/README.md
index 709857a..7755e95 100644
--- a/README.md
+++ b/README.md
@@ -12,7 +12,7 @@ SQLCoder-70B and SQLCoder-34B are fine-tuned on a base CodeLlama model.
 | model   | perc_correct |
 |-|-|
 | defog-sqlcoder-70b-alpha    | 93.0 |
-| defog-sqlcoder-7b-2    | 87.0 |
+| defog-sqlcoder-7b-2    | 90.5 |
 | defog-sqlcoder-34b-alpha    | 84.0 |
 | gpt4-2024-01-30       | 82.0 |
 | gpt4-turbo-2024-01-30 | 78.0 |
@@ -36,15 +36,17 @@ You can read more about our [training approach](https://defog.ai/blog/open-sourc
 
 ## Results by question category
 We classified each generated question into one of 6 categories. The table displays the percentage of questions answered correctly by each model, broken down by category.
-|               | date | group_by | order_by | ratio | join | where |
-| ------------- | ---- | -------- | -------- | ----- | ---- | ----- |
-| sqlcoder-70b  | 96   | 91.4     | 97.1     | 85.7  | 97.1 | 91.4  |
-| sqlcoder-34b  | 80   | 94.3     | 85.7     | 77.1  | 85.7 | 80    |
-| gpt-4         | 64   | 94.3     | 88.6     | 74.2  | 85.7 | 80    |
-| sqlcoder2-15b | 76   | 80       | 77.1     | 60    | 77.1 | 77.1  |
-| sqlcoder-7b   | 64   | 82.9     | 74.3     | 54.3  | 74.3 | 74.3  |
-| gpt-3.5       | 68   | 77.1     | 74.2     | 34.3  | 65.7 | 71.4  |
-| claude-2      | 52   | 71.4     | 74.3     | 57.1  | 65.7 | 62.9  |
+|                | date | group_by | order_by | ratio | join | where |
+| -------------- | ---- | -------- | -------- | ----- | ---- | ----- |
+| sqlcoder-70b   | 96   | 91.4     | 97.1     | 85.7  | 97.1 | 91.4  |
+| sqlcoder-7b    | 96   | 91.4     | 94.3     | 91.4  | 94.3 | 77.1  |
+| sqlcoder-34b   | 80   | 94.3     | 85.7     | 77.1  | 85.7 | 80    |
+| gpt-4          | 72   | 94.3     | 97.1     | 80    | 91.4 | 80    |
+| gpt-4-turbo    | 76   | 91.4     | 91.4     | 62.8  | 88.6 | 77.1  |
+| natural-sql-7b | 56   | 88.6     | 85.7     | 60    | 88.6 | 80    |
+| sqlcoder-7b    | 64   | 82.9     | 74.3     | 54.3  | 74.3 | 74.3  |
+| gpt-3.5        | 72   | 77.1     | 82.8     | 34.3  | 65.7 | 71.4  |
+| claude-2       | 52   | 71.4     | 74.3     | 57.1  | 65.7 | 62.9  |
 
 ## Using SQLCoder
 You can use SQLCoder via the `transformers` library by downloading our model weights from the Hugging Face repo. We have added sample code for [inference](./inference.py) on a [sample database schema](./metadata.sql).