Dynamically add table aliases without an LLM + de-duplicate columns in pandas

[rishsriv]

This automatically generates relevant table aliases and appends it to a prompt. Doing so transfers the onus of creating table aliases away from the LLM. We may have to retrain our LLM to expect this kind of prompting, so that it expects a more varied source of inputs.

Here's an example of how to run this.

python main.py \
-db postgres \
-q "data/questions_gen_postgres.csv" "data/instruct_basic_postgres.csv" "data/instruct_advanced_postgres.csv" "data/idk.csv" \
-o results/classic_new_reprompt.csv results/basic_new_reprompt.csv results/advanced_new_reprompt.csv results/idk_new_reprompt.csv \
-g api \
-b 1 \
-f prompts/prompt_cot.md \
--api_url "YOUR_API_ENDPOINT" \
--api_type "vllm" \
-p 10 \
-c 0 --logprobs --cot_table_alias

[wongjingping]

Thanks for the deduplicating fix and the updates! 2 small comments

[wongjingping]

nit: shall we add a separate argument in generate_prompt, say cot_pregen, to tack on the table aliases? I was thinking cot_instructions would be more of the "instructions" of what to do, while cot_pregen would be the actual output (eg table aliases)

[rishsriv]

Aye sounds good!

[wongjingping]

Shall we keep the cot_instructions field here? This is more for letting the model know that it should be generating the aliases followed by the SQL.
Currently in the data we use the instruction to let the model know when it should generate the table aliases, and when it should just directly generate the SQL (to facilitate the mixing of both types of data).

[rishsriv]

Sounds good – making a fix now!

[rishsriv]

Fixed! We can now use --cot_table_alias instruct to get the model to use {cot_instructions}, and --cot_table_alias pregen to pre-generate table aliases