clean cURL commands

Remove the bloat

clion | 2021-04-17
At home

Often times you'll use Firefox's devtools to turn a network request into a curl command:

Which results in the following on the clipboard:

curl 'http://natas11.natas.labs.overthewire.org/' 
-H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0' 
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' 
-H 'Accept-Language: en-US,en;q=0.5' 
-H 'Accept-Encoding: gzip, deflate' 
-H 'DNT: 1'
-H 'Authorization: Basic foo==' 
-H 'Connection: keep-alive' 
-H 'Upgrade-Insecure-Requests: 1' 
-H 'Pragma: no-cache' 
-H 'Cache-Control: no-cache'

This is often unnecessarily verbose. During CTF, you'll probably only need:

  1. The URL
  2. Cookies and/or Authorization header
  3. POST payload (if present)

Silly script

We made a small tool that cleans the command up a bit. The result will be:

curl -vvk -H 'Authorization: Basic foo==' 'http://natas11.natas.labs.overthewire.org/'

The script will keep the authorization header, as well as cookies, POST data, and the HTTP method.

Disclaimer: the code is quite bad - but oh well.

Usage is just running the program without arguments.

#!/usr/bin/python3
# sudo apt install -y xsel
import re, sys
from subprocess import Popen, PIPE, DEVNULL, STDOUT

print("Cleans a cURL command. Enter command below")
inp = input().replace("\n", ""),replace("\\\n","").strip()
if not inp.startswith("curl "):
    raise Exception("input must start with 'curl '\n")

inp = inp[5:]
cmd = ["curl", "-vvk"]
for kv in re.findall(R"( \-\w \'(.+?)\')", inp):
    inp = inp.replace(kv[0], "")
    for needle in ["authorization:", "cookie:", " -d ", " -b ", " -X"]:
        if needle in kv[0].lower():
            cmd.append(kv[0].strip()) 
cmd.append(inp)
result = " ".join(cmd)

p = Popen(['xsel', '-b'], stdin=PIPE, stdout=DEVNULL, stderr=STDOUT)
p.communicate(input=result.encode())

sys.stderr.write("\nCopied cleaned cURL command to clipboard\n")

Closing note

What is also handy is automatically converting cURL commands to Python, PHP, NodeJS code using the following website: curlconverter.com