gptq - my basic command-line glue

14 Mar, 2023

I've had some moderate success with a ChatGPT prompt that helps me reshape structured data into another structured format. I coded up a little command-line tool: gptq and pushed it up so that y'all can have a play with it too.

It's limited in the amount of data it can process, but I found it pretty useful as a sort of glue-like program that can pipe data between two programs pretty effortlessly.

Here's some stuff I've tried.

I took a few lines from this dataset and asked it to convert it JSON. Worked ok.

cat movies.csv | gptq 'convert to json objects' -f json | jq

[
  {
    "IMDb-rating": "4.8",
    "appropriate_for": "R",
    "director": "John Swab",
    "downloads": "304",
    "id": "372092",
    "industry": "Hollywood / English",
    "language": "English",
    "posted_date": "20 Feb, 2023",
    "release_date": "Jan 28 2023",
    "run_time": "105",
    "storyline": "\"Doc facilitates a fragile truce between the Governor and Cartel, trading prosecutorial leniency for finance. With no more truce, Doc is left to fend for himself and protect the one untainted thing in his life: his daughter, Little Dixie.\"",
    "title": "Little Dixie",
    "views": "2,794",
    "writer0": "John Swab"
  },
  {
    "IMDb-rating": "6.4",
    "appropriate_for": "TV-PG",
    "director": "Paul Ziller",
...

I scanned some websites. Kind of boring, but could be cool if I could figure out a better prompt.

cat ips.txt | httpx -silent -json -tech-detect | gptq 'list host, ip, and running webserver' -f csv

107.182.233.166, https://107.182.233.166, nginx/1.22.1
151.101.66.194, https://151.101.66.194, Varnish

I tried basic sentiment analysis... it was ok. Lots of neutral titles. Don't know if that's how Hacker News is or it's something going on with ChatGPT.

curl --silent https://news.ycombinator.com/ | htmlq '.titleline' --text | gptq 'title and brief sentiment on title'

GPT-4 (openai.com) - Neutral
Was there a tech-hiring bubble? Job postings data suggest so (stlouisfed.org) - Neutral
Two U.S. Men Charged in 2022 Hacking of DEA Portal (krebsonsecurity.com) - Negative
Kali Linux 2023.1 introduces 'Purple' distro for defensive security (gitlab.com/kalilinux) - Neutral
My Startup Banking Story (mitchellh.com) - Neutral
Online Multiplayer on the Game Boy [video] (youtube.com) - Neutral
Live-caption glasses let deaf people read conversations [video] (youtube.com) - Positive
Show HN: Whatdoesthiscodedo.com – AI explanations for other people’s code (whatdoesthiscodedo.com) - Neutral
40% of the code GitHub Copilot users check-in is AI generated and unmodified (microsoft.com) - Neutral
...

Movie recs! I fed it my most recent movies I logged from my Letterboxd and tried to get some recommendations out of it. I didn't have too much luck getting good recommendations, but it did return some movies so...

curl --silent https://letterboxd.com/willmeyers/films/diary/ | htmlq '.headline-3' --text | gptq 'a newline sep list recommending some movies like these'

The Bourne Identity
The Departed
The Dark Knight

I watched Die Hard at Christmas and it was still in my diary 😐

Overall, a bit underwhelming. But maybe we'll see some improvement with GPT-4.