Skip to content

Fix input handling around -P flag with the DNS module, Resolves #746#757

Merged
phillip-stephens merged 20 commits intomainfrom
phillip/746
Feb 5, 2024
Merged

Fix input handling around -P flag with the DNS module, Resolves #746#757
phillip-stephens merged 20 commits intomainfrom
phillip/746

Conversation

@phillip-stephens
Copy link
Contributor

@phillip-stephens phillip-stephens commented Dec 15, 2023

Context available in Issue #746

  • added new parsing to identify how many DNS questions in --probe-args the user passed in based on count of semi-colons
  • error out if user uses a value for -P that isn't a multiple of the number of DNS questions
    • Ex: -P 4 --probe-args="A,google.com;AAAA,cloudflare.com" is acceptable since 4 probes is a multiple of 2 DNS questions

Note: The --dedup uses the scanned host's source IP address to de-duplicate responses. This means that if you send 4 probes and 2 DNS queries to 8.8.8.8 (2 probes per question), then ZMap only marks 1 of the 4 as a "application success". This seems mis-leading, but perhaps this is just how ZMap is designed? The de-dupeing is baked into recv.c, so it's not probe-module specific.

Tests

  • Tested with no probe args - ``
  • Tested with 1 arg - A,google.com
  • Tested with 1 arg, leading semicolon - ;A,google.com
  • Tested with 1 arg, trailing semicolon - A,google.com;
  • Tested with 2 arg - A,google.com;AAAA,yahoo.com
  • Tested with bad domain - A,

Notice how there's 2 responses in the logfile.json, and that the app hitrate being printed maxes out at 50%

pstephens@scratch-04-a:~/zmap-rsync$ sudo ./src/zmap -p 53 --probe-module=dns -O json -o logfile.json -P 2 1.1.1.1 --sender-threads=1 --probe-args="A,yahoo.com" --output-fields=* -c 3
Dec 19 19:08:37.852 [INFO] dedup: Response deduplication method is full
Dec 19 19:08:37.852 [INFO] filter: No output filter provided. ZMap will output all results, including duplicate and non-successful responses (e.g., RST and ICMP packets). If you want a filter similar to ZMap's default behavior, you can set an output filter similar to the following: --output-filter="success=1 && repeat=0".
Dec 19 19:08:37.907 [INFO] recv: duplicate responses will be passed to the output module
Dec 19 19:08:37.907 [INFO] recv: unsuccessful responses will be passed to the output module
 0:00 2%; sent: 2 done (42 p/s avg); recv: 0 0 p/s (0 p/s avg); app success: 0 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 0.00% app hitrate: 0.00%
 0:00 2%; sent: 2 done (42 p/s avg); recv: 0 0 p/s (0 p/s avg); app success: 0 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 0.00% app hitrate: 0.00%
 0:01 34%; sent: 2 done (42 p/s avg); recv: 2 2 p/s (1 p/s avg); app success: 1 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00% app hitrate: 50.00%
 0:02 67%; sent: 2 done (42 p/s avg); recv: 2 0 p/s (0 p/s avg); app success: 1 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00% app hitrate: 50.00%
 0:03 101%; sent: 2 done (42 p/s avg); recv: 2 0 p/s (0 p/s avg); app success: 1 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00% app hitrate: 50.00%
Dec 19 19:08:41.950 [INFO] zmap: completed
pstephens@scratch-04-a:~/zmap-rsync$ cat logfile.json
{"saddr":"1.1.1.1","saddr_raw":16843009,"daddr":"171.67.71.40","daddr_raw":675759019,"ipid":64826,"ttl":57,"sport":53,"dport":59554,"classification":"dns","success":true,"app_success":true,"udp_len":131,"dns_id":28666,"dns_rd":1,"dns_tc":0,"dns_aa":0,"dns_opcode":0,"dns_qr":1,"dns_rcode":0,"dns_cd":0,"dns_ad":0,"dns_z":0,"dns_ra":1,"dns_qdcount":1,"dns_ancount":6,"dns_nscount":0,"dns_arcount":0,"dns_questions":[{"name":"yahoo.com","qtype":1,"qtype_str":"A","qclass":1}],"dns_answers":[{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1065,"rdlength":4,"rdata_is_parsed":1,"rdata":"74.6.143.26"},{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1065,"rdlength":4,"rdata_is_parsed":1,"rdata":"98.137.11.163"},{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1065,"rdlength":4,"rdata_is_parsed":1,"rdata":"74.6.143.25"},{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1065,"rdlength":4,"rdata_is_parsed":1,"rdata":"98.137.11.164"},{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1065,"rdlength":4,"rdata_is_parsed":1,"rdata":"74.6.231.20"},{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1065,"rdlength":4,"rdata_is_parsed":1,"rdata":"74.6.231.21"}],"dns_authorities":[],"dns_additionals":[],"dns_parse_err":0,"dns_unconsumed_bytes":0,"raw_data":"6ffa81800001000600000000057961686f6f03636f6d0000010001c00c000100010000042900044a068f1ac00c0001000100000429000462890ba3c00c000100010000042900044a068f19c00c0001000100000429000462890ba4c00c000100010000042900044a06e714c00c000100010000042900044a06e715","repeat":false,"cooldown":true,"timestamp_str":"2023-12-19T19:08:38.894+0000","timestamp_ts":1703012918,"timestamp_us":894721}
{"saddr":"1.1.1.1","saddr_raw":16843009,"daddr":"171.67.71.40","daddr_raw":675759019,"ipid":12105,"ttl":57,"sport":53,"dport":59553,"classification":"dns","success":true,"app_success":true,"udp_len":131,"dns_id":28666,"dns_rd":1,"dns_tc":0,"dns_aa":0,"dns_opcode":0,"dns_qr":1,"dns_rcode":0,"dns_cd":0,"dns_ad":0,"dns_z":0,"dns_ra":1,"dns_qdcount":1,"dns_ancount":6,"dns_nscount":0,"dns_arcount":0,"dns_questions":[{"name":"yahoo.com","qtype":1,"qtype_str":"A","qclass":1}],"dns_answers":[{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1672,"rdlength":4,"rdata_is_parsed":1,"rdata":"74.6.143.25"},{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1672,"rdlength":4,"rdata_is_parsed":1,"rdata":"74.6.231.21"},{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1672,"rdlength":4,"rdata_is_parsed":1,"rdata":"98.137.11.163"},{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1672,"rdlength":4,"rdata_is_parsed":1,"rdata":"74.6.143.26"},{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1672,"rdlength":4,"rdata_is_parsed":1,"rdata":"98.137.11.164"},{"name":"yahoo.com","type":1,"type_str":"A","class":1,"ttl":1672,"rdlength":4,"rdata_is_parsed":1,"rdata":"74.6.231.20"}],"dns_authorities":[],"dns_additionals":[],"dns_parse_err":0,"dns_unconsumed_bytes":0,"raw_data":"6ffa81800001000600000000057961686f6f03636f6d0000010001c00c000100010000068800044a068f19c00c000100010000068800044a06e715c00c0001000100000688000462890ba3c00c000100010000068800044a068f1ac00c0001000100000688000462890ba4c00c000100010000068800044a06e714","repeat":true,"cooldown":true,"timestamp_str":"2023-12-19T19:08:38.895+0000","timestamp_ts":1703012918,"timestamp_us":895146}

@phillip-stephens phillip-stephens added this to the ZMap 4.1 milestone Dec 15, 2023
@phillip-stephens phillip-stephens linked an issue Dec 15, 2023 that may be closed by this pull request
@phillip-stephens phillip-stephens marked this pull request as ready for review December 15, 2023 20:37
@zakird
Copy link
Member

zakird commented Dec 16, 2023

I don't think that I'm fully parsing how this is supposed to work. I would expect conf->packet_streams to be set to a different value based on the description of the PR, but I don't actually see that logic anywhere (i.e., I don't see the number of probes sent ever changed by the probe module).

That said, I don't love the idea of probe modules changing global configuration — that seems very messy to me. I think it would make sure sense to instead throw a log_fatal and explain what the user needs to do.

Last note, I could imagine that someone might want to send a multiple of the number of probes (e.g., if there are two names, and you want to send a packet to each twice, then you set the number of probes to be 4).

@phillip-stephens
Copy link
Contributor Author

phillip-stephens commented Dec 19, 2023

@zakird You're right, I've updated the file to allow the user to use a multiple of the DNS question count as the probe number. Added details in the PR description.

WDYT about the app hitrate that's logged only counting a single response from a single IP as a success? I know that behavior makes sense with a TCP scan where you only have one "flavor" of probe sent to any given IP, but here we sorta have however many as there are DNS questions.
Example:

pstephens@scratch-04-a:~/zmap-rsync$ sudo ./src/zmap -p 53 --probe-module=dns -O json -o logfile.json -P 2 1.1.1.1 --sender-threads=1 --probe-args="A,yahoo.com;AAAA,google.com" --output-fields=* -c 3
Dec 19 19:14:24.923 [INFO] dedup: Response deduplication method is full
Dec 19 19:14:24.923 [INFO] filter: No output filter provided. ZMap will output all results, including duplicate and non-successful responses (e.g., RST and ICMP packets). If you want a filter similar to ZMap's default behavior, you can set an output filter similar to the following: --output-filter="success=1 && repeat=0".
Dec 19 19:14:24.978 [INFO] recv: duplicate responses will be passed to the output module
Dec 19 19:14:24.978 [INFO] recv: unsuccessful responses will be passed to the output module
 0:00 0%; sent: 0 0 p/s (0 p/s avg); recv: 0 0 p/s (0 p/s avg); app success: 0 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 0.00% app hitrate: 0.00%
 0:00 0%; sent: 0 0 p/s (0 p/s avg); recv: 0 0 p/s (0 p/s avg); app success: 0 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 0.00% app hitrate: 0.00%
 0:01 34%; sent: 2 done (39 p/s avg); recv: 2 2 p/s (1 p/s avg); app success: 1 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00% app hitrate: 50.00%
 0:02 67%; sent: 2 done (39 p/s avg); recv: 2 0 p/s (0 p/s avg); app success: 1 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00% app hitrate: 50.00%
 0:03 101%; sent: 2 done (39 p/s avg); recv: 2 0 p/s (0 p/s avg); app success: 1 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00% app hitrate: 50.00%
Dec 19 19:14:29.022 [INFO] zmap: completed

Notice the app hitrate at 50% even with the probes being unique from a DNS perspective.

Seems it'd be a bit confusing to users if they didn't go look at the output file.

Copy link
Member

@zakird zakird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this seems fine, but personally, would have tried to have the string parsing logic in one place in the probe module instead of having counting in one place and parsing in another place such that both always have to be updated. There's a comment inline about this.

// find how many probe_args the user wants to query
if (conf->probe_args) {
int arg_strlen = strlen(conf->probe_args);
for (int i = 0; i < arg_strlen; i++) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the more canonical way to do this would be to use strtok or strtok. More broadly though, I do wonder if it makes sense that this is logic is different from the logic we use for actually parsing out the questions. It seems that when parsing, we would know this information already and then wouldn't have this logic in different places.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair, and thanks for the suggestion to look at stroke, seems very much like the correct way to go about this. I've re-done the logic and it seems much cleaner. I've re-tested the scenarios I outlined in the PR, mind taking a second look and we can merge if it looks good to you? @zakird

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: let me integrate the changes made here, they'll conflict.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll re-assign you when done.

@zakird zakird removed their assignment Jan 15, 2024

const char *qopts_rn = "rn";
static uint8_t *rdbits;
const char *qopts_rn = "nr"; // used in query to disable recursion bit in DNS header
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated this, realized that we're using nr in the --help text, and I think it makes more sense as an abbreviation for no recursion

@phillip-stephens
Copy link
Contributor Author

Ready for a re-review, logic is all in one place now @zakird

Copy link
Member

@zakird zakird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks sane, but I could 100% also have missed something here. Assuming that you have tested though and it works, LGTM to merge.

@phillip-stephens phillip-stephens merged commit a9d7036 into main Feb 5, 2024
@phillip-stephens phillip-stephens deleted the phillip/746 branch February 5, 2024 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problem with -P option in DNS module

2 participants