How to query CSVs with SQLite
I’ve got the following shell function in my shell startup file that allows me to quickly query a CSV file with SQL:
cq() {
local csv_file="${1:?Usage: cq <csv_file> [<table> [<sql>]]}" table="${2:-T}" sql="${3:-"-interactive"}"
sqlite3 :memory: -header -csv -cmd ".import '$csv_file' '$table'" "$sql"
}
I can’t claim to have written this function. I found it on this GitHub issue thread. I’ve just tweaked it a little to add an optional SQL query argument, and used some more standard SQLite parameters.
Usage
First argument is the CSV file to query. Second argument is the name of the table to create in the SQLite database. Third argument is the SQL query to run. If you don’t specify a query then you’ll be dropped into an interactive SQLite shell.
cq <(curl -s https://raw.githubusercontent.com/owid/covid-19-data/master/public/data/vaccinations/vaccinations.csv) \
vax \
"SELECT location, date, total_vaccinations FROM vax WHERE location = 'United Kingdom' ORDER BY date DESC LIMIT 10;"
Alternatives
If you’re looking for something a bit more powerful then there are dozens of tools out there that allow you to query CSV files. Here are a few of the popular ones: