Skip to content

10 · Code analysis, bytes, and reverse-engineering foundations

Depth: developed lesson · Prerequisites: Linux and networking, evidence and reporting · Time: three 90-minute sessions.

A parser turns untrusted bytes into a structured interpretation. Before investigating a complex executable, learn to say which bytes support which interpretation, which assumptions the code makes, and which inputs contradict them. This chapter supplies a tiny original format whose entire boundary can be examined.

The fixture is data processing, not malware. Compiled-binary disassembly, debugging, memory corruption, unpacking and malware behavior analysis remain later material in the coverage map. Running this decoder does not establish competence in those subjects.

Outcomes

  • Map byte offsets to fields and explain byte order.
  • Trace input through validation into a typed result.
  • Predict valid, truncated, unsupported and trailing-byte cases.
  • Separate recognizing an operation from executing it.
  • Use bounded mutation tests and explain their coverage limits.

Session 1: reverse a tiny documented format

The fictional DW frame version 1 has a six-byte header followed by an operation-specific body. All multi-byte integers are unsigned and big-endian. Its maximum accepted size is 256 bytes; the two defined operations are much smaller.

Offset, zero-based Width Field Contract
0 2 bytes Magic ASCII DW, hex 44 57
2 1 byte Version Exactly 01
3 1 byte Opcode 01 = read; 02 = write
4 2 bytes Body length Counts only bytes after the header
6 2 bytes Object ID Integer from 0 through 65535
8 2 bytes, write only Value Integer from 0 through 65535

This is not Modbus or a real device command. A decoded “write” is a Python data object; the decoder has no resource store, socket or execution engine.

Work through this frame before running a command:

44 57 | 01 | 01 | 00 02 | 00 65
magic   ver  op   length  object

The body length is two. The object bytes encode 0x0065, or decimal 101. Big-endian places the most significant byte first: 01 00 is 256, while 00 01 is 1. Python's struct format prefix > selects big-endian standard sizes without native padding. Explicit layout avoids depending on the host's native layout. Python struct documentation.

Now decode 44 57 01 02 00 04 00 ca 00 07 on paper. Name the operation, object ID and value. Which field tells you that there must be four body bytes?

Worked interpretation

Version 1, opcode 2: write; declared body length 4; object 202; value 7. The header length and the opcode's body contract agree. This establishes an interpretation of supplied bytes, not an actual write or authorization decision.

Inspect the implementation before running it

Open labs/binary_frame/__main__.py. Follow from_hex into decode, then into the immutable Frame result. Find each boundary:

  1. The text input is bounded before converting hexadecimal to bytes.
  2. The frame must be large enough for a header and below the teaching cap.
  3. Magic, version and opcode must be supported.
  4. Declared length must equal the available body length.
  5. The body size must match the selected operation.
  6. Only then are the body integers interpreted.

These checks answer different questions. A frame may contain enough bytes overall yet still contain the wrong body size for its opcode. A valid integer field does not establish whether an actor is allowed to request that object. Parsing and authorization are separate stages.

Session 2: compare predictions with execution

From the repository root, with Python 3.12 or newer:

python3 -m labs.binary_frame
python3 -m labs.binary_frame --hex 4457010100020065
python3 -m labs.binary_frame --hex 44570102000400ca0007

The default exercise prints six cases: two decoded and four rejected. The individual valid commands return a read of object 101 and a write-shaped record for object 202, value 7. No files are changed and no requests are sent.

Try these malformed cases individually. Predict which invariant fails before executing each command:

python3 -m labs.binary_frame --hex 44570101000200
python3 -m labs.binary_frame --hex 445701010002006500
python3 -m labs.binary_frame --hex 4457010900020065
Expected rejection reasons

The first claims two body bytes but contains one. The second claims two but contains three. The third uses unsupported opcode 9. Each individual command exits with status 2 and an explicit rejection. The default six-case report catches these expected per-case errors and continues so you can compare them together.

Why reject an extra byte instead of ignoring it? This interface accepts exactly one frame. Accepting a valid prefix and silently discarding a suffix would conceal data from the caller. A streaming protocol would need an explicit contract for consumed bytes, buffering and additional frames; this decoder does not provide that contract.

Session 3: a bounded mutation experiment

Run this original exercise from the repository root. It changes only in-memory bytes:

from labs.binary_frame.__main__ import decode

original = bytes.fromhex('4457010100020065')
rejected = 0
for size in range(len(original)):
    try:
        decode(original[:size])
    except ValueError:
        rejected += 1
print('rejected truncations:', rejected)

objects = set()
for low_byte in range(256):
    frame = original[:-1] + bytes([low_byte])
    objects.add(decode(frame).object_id)
print('distinct decoded object IDs:', len(objects))

Expect eight rejected truncations and 256 distinct object IDs. Changing a data byte can produce another valid frame; “the input changed” does not imply “the parser should fail.” Conversely, all prefixes of this complete read frame violate its contract.

This is a finite, reproducible mutation experiment, not exhaustive fuzzing. It covers eight truncations and one byte's possible values. It does not explore every header combination, parser implementation, compiler or memory-safety condition. The accompanying tests also check unsupported versions, opcode/body mismatches and invalid hex.

What static evidence can establish

Finding a string such as a URL or operation name in a file is an observation about bytes. It does not prove that an executable reaches that code path or sends a request. Here the string write appears in the source and output, but the program does not perform writes to an application object. Inspect calls and data flow before translating a label into a behavioral claim.

For unknown samples, record identity, provenance and static observations before considering a separate isolated execution environment. This supplied parser is known course code; do not replace it with an executable found in a book archive.

Evidence and acceptance

Keep the offset table annotated with both worked frames, the six-case output, three rejection predictions and the mutation results. Acceptance requires explaining both why malformed frames fail and why the 256 object variants remain valid. Attach the repository commit so a reviewer can inspect the exact parser used.

Safety and scope

The lab only interprets bounded synthetic byte strings. It does not execute extracted commands or unknown binaries. There is no persistent state to reset. Keep any private exercise script separate from the repository's original fixture.

Teach-back

  1. Why must byte order and the length denominator be explicit?
  2. Which checks would a valid-prefix-only parser miss?
  3. What separates a decoded operation from an authorized or executed action?
  4. Which untested input family would you examine next, and why?

Continue: 14 · Cryptography and secure engineering, then compare this framing exercise with industrial protocol analysis.