Integer Conversion and Overflow

Bug Classes and PrimitivesLesson 5 of 860 minutesLessonNo lab next

The direction the last bug couldn’t go

In Out-of-Bounds Writes and Variable Layout the decision variable sat four bytes below the buffer, and strcpy only runs forward, so no input of any length reached it. The write had a direction and the target was on the wrong side of it.

An indexed write can reach in either direction, provided the check lets the index through. We’ll start with a check that rejects numbers that are too large but says nothing about negative ones, then follow what happens when a checked number changes type before it is used. In both cases, the code contains a check; the question is whether that check constrains the operation that actually follows.

Learning objectives

By the end of this lesson

• Get a write past a bounds check that rejects everything it was written to reject.
• Work out which index reaches a chosen neighbour, rather than trying values until one does.
• Explain what a signed value does to a comparison and to a length.
• Recognise the single-comparison idiom that closes this, and say why it works.

A board with a bounds check

In the Ubuntu terminal, create the working directory if needed and move into it:

mkdir -p ~/underdunn-course/bug-classes ~/underdunn-course/notes
cd ~/underdunn-course/bug-classes

Save the complete program below as consignment.c. Keep the terminal in this directory for the build commands; no source file from an earlier lesson is needed.

/*
 * A consignment board. Slots hold quantities; the board records whether
 * the consignment has been cleared for collection.
 */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define SLOTS 8

static int cleared = 0;
static int slots[SLOTS];

static void store(int slot, int value)
{
    if (slot >= SLOTS) {
        puts("  rejected: no such slot");
        return;
    }
    slots[slot] = value;
    printf("  slot %d = %d\n", slot, value);
}

static void board(void)
{
    printf("  slots:");
    for (int i = 0; i < SLOTS; i++)
        printf(" %d", slots[i]);
    printf("   cleared=%d %s\n", cleared,
           cleared ? "[COLLECTION AUTHORISED]" : "");
}

int main(int argc, char **argv)
{
    setvbuf(stdout, NULL, _IONBF, 0);

    for (int i = 1; i + 1 < argc; i += 2)
        store(atoi(argv[i]), atoi(argv[i + 1]));

    board();
    return 0;
}

gcc -g -O0 -fno-omit-frame-pointer -Wall -Wextra consignment.c -o consignment
./consignment 0 11 1 22 7 99

  slot 0 = 11
  slot 1 = 22
  slot 7 = 99
  slots: 11 22 0 0 0 0 0 99   cleared=0

Eight slots, and store checks the slot number before using it. Try to get past it the way the depot’s check was got past in Lesson 3:

./consignment 8 1
./consignment 99 1

  rejected: no such slot
  rejected: no such slot

The check holds. It was written to stop a slot number that is too large, and it stops every slot number that is too large. But what about -1?

The other end

./consignment -1 1

  slot -1 = 1
  slots: 0 0 0 0 0 0 0 0   cleared=0

Accepted. -1 >= 8 is false, so nothing objected, and the write went somewhere four bytes before slots.

The board doesn’t show a changed slot or flag, but that doesn’t mean the write was rejected. Its own message says it stored at index -1, which is outside the eight values board prints. We can use the symbol addresses to work out what lies before the array instead of treating an unchanged display as a failed write.

Which index reaches cleared

The arithmetic is the one from Out-of-Bounds Reads, with a sign on it:

for s in cleared slots; do
  printf '%-9s %s\n' "$s" "$(nm ./consignment | awk -v s=$s '$3==s{print $1}')"
done

cleared   0000000000004040
slots     0000000000004060

slots begins at 0x4060 and cleared is at 0x4040, which is thirty-two bytes earlier. Each element is four bytes, so cleared is element -8, and the thirty-two byte gap is why -1 did nothing you could see. It landed in alignment padding between the two.

./consignment -8 1

  slot -8 = 1
  slots: 0 0 0 0 0 0 0 0   cleared=1 [COLLECTION AUTHORISED]

The value is yours as well, not just the location:

./consignment -8 4242 0 7

  slot -8 = 4242
  slot 0 = 7
  slots: 7 0 0 0 0 0 0 0   cleared=4242 [COLLECTION AUTHORISED]

Why the check misses

slot is an int, SLOTS is 8, and slot >= SLOTS compares two signed values, which is exactly what it looks like it does. Negative numbers are smaller than eight, so they pass. The check has one bound and the array has two.

The direct repair is slot < 0 || slot >= SLOTS: reject either side of the valid interval. You may also encounter a version that uses an unsigned conversion to do both jobs with one comparison:

if ((unsigned)slot >= SLOTS) { /* reject */ }

slot	slot >= SLOTS rejects	(unsigned)slot >= SLOTS rejects
0	no	no
7	no	no
8	yes	yes
-1	no	yes
-3	no	yes
99	yes	yes

Casting -1 to unsigned gives 4294967295 on this platform, where unsigned int is 32 bits. Every negative int converts to a value above our small bound of eight, while zero through seven retain their values. The comparison therefore rejects both ends.

Try either repair in a copy named consignment-fixed.c, build it as consignment-fixed, and rerun the valid cases and negative indices. The valid stores should still work, but neither negative request should reach the assignment.

When the checked number changes

The unsigned conversion helped that check, but the same kind of conversion can cause trouble if it happens after a different check. Create a separate conversions.c so we can see the numbers without attempting an enormous copy or allocation:

#include <stdio.h>
#include <stddef.h>

int main(void)
{
    int length = -1;
    short requested = 300;
    unsigned int count = 0x40000000u;
    unsigned int bytes = count * 4u;

    printf("signed check accepts -1: %d\n", length <= 16);
    printf("length passed as size_t: %zu\n", (size_t)length);
    printf("requested: %d, narrowed: %d\n", requested,
           (signed char)requested);
    printf("count: %u, bytes: %u\n", count, bytes);
    return 0;
}

gcc -std=c11 -g -O0 -Wall -Wextra conversions.c -o conversions
./conversions

On the course’s x86-64 GCC build:

signed check accepts -1: 1
length passed as size_t: 18446744073709551615
requested: 300, narrowed: 44
count: 1073741824, bytes: 0

A signed length becomes unsigned

Start with length. The comparison uses two signed integers, so -1 <= 16 is true.

size_t is the unsigned integer type C uses for object sizes and many byte-count arguments. A later call expecting a size_t, such as a copy function’s length argument, converts that same -1 to the maximum value of the unsigned destination type. The check accepted a small negative number; the called function receives a huge positive length. No arithmetic overflow was needed.

The types in the comparison matter too. sizeof buffer has type size_t, not int, so substituting it for the literal 16 on this platform converts the negative operand during the comparison itself.

The operator alone doesn’t tell us what happens. We need the types on both sides of the check, and the type used by the eventual access.

A value no longer fits its type

Next, requested fits in a short but not in an eight-bit signed char. GCC’s conversion leaves 44, so code that validates 300 and later stores it in that smaller type is no longer using the number it checked.

This result is documented by GCC’s integer-conversion rules. Out-of-range conversion to a signed type is implementation-defined in the C11 mode used here, so 44 isn’t a result to promise for every compiler.

A multiplication wraps before assignment

Finally, count * 4u is evaluated as 32-bit unsigned arithmetic on this build. The mathematical product is 4294967296, one beyond the largest representable value, so it wraps to zero. Assigning that already-wrapped result to a wider type afterwards would not recover the missing bits.

If an allocation uses bytes while a loop still uses count, the two operations disagree about how much storage was requested.

Compare another set of values

Before changing the program, predict the output for length = -2, requested = 257, and count = 0x40000001u. Then run those values and identify which check or calculation would need to happen before a real copy. Keep this probe non-copying: the point is to inspect the length that would reach the operation, not to allocate gigabytes.

Compare the changed values

On this build:

• The signed check still accepts -2, which becomes 18446744073709551614 as size_t. The length needs a lower bound before conversion.
• The narrowed value is 1. A value needs to fit its destination type, not just the type it had when checked.
• The unsigned product is 4. Multiplication needs a check that the product will fit before it is evaluated.

These are three different failure points, even though all three can eventually produce an undersized buffer or an oversized access.

What the sanitizer sees

gcc -g -O0 -fsanitize=address consignment.c -o consignment-asan
./consignment-asan -1 1

==13952==ERROR: AddressSanitizer: global-buffer-overflow
WRITE of size 4 at 0x642d7fc0f2dc thread T0
0x642d7fc0f2dc is located 56 bytes after global variable 'cleared' defined in 'consignment.c:11:12' of size 4
0x642d7fc0f2dc is located 4 bytes before global variable 'slots' defined in 'consignment.c:12:12' of size 32

It reports the -1 write as a WRITE four bytes before slots. The distances also show that this instrumented build laid the globals out differently from the plain one. The report confirms the invalid access, but its offsets aren’t replacements for the plain binary’s nm table.

An access outside an array has undefined behavior in C, meaning the language no longer promises a particular result. The plain build’s observed neighbor is useful evidence about that build, not a portable guarantee that another compiler must place or access the objects the same way.

There is no mention of a signed comparison or a missing lower bound in that diagnostic. It identifies the invalid access; we still have to read the condition that allowed it.

Key concept

The access and its cause

The out-of-bounds write and the missing lower bound describe different parts of the same failure. The check runs exactly as written, but it only answers half the question the array needs answered.

Signed comparisons aren’t inherently wrong. Adding the missing lower bound repairs this condition without changing the type, while the conversion examples need their own checks at the point where a value changes meaning.

Knowledge check

Question 1 of 3

Why did ./consignment -1 1 change nothing visible, when -8 authorised collection?

Index -1 was rejected by the check and never wrote anythingIndex -1 wrote into alignment padding between cleared and slotsNegative indices are undefined behaviour and the compiler removed the write

Check answer

When the results don’t match

• If -8 lands somewhere harmless, your toolchain placed the globals differently. Read your own nm output, take the difference, divide by four, and use that index instead.
• If -1 faults immediately, first check which binary you ran. The instrumented build is expected to report the invalid access, while the plain build’s behavior depends on where that access lands. Neither build makes the negative index valid.
• If the naive and idiomatic checks agree on every case, confirm SLOTS is an int constant rather than something already unsigned. A size_t bound makes the comparison unsigned on its own and the lesson’s point disappears.
• If large negative indices crash where the transcript shows them accepted, that is a difference in what happens to be mapped below the data segment and not an error.

To recover a known state, rebuild from the listing. The board keeps nothing between runs.

Check your understanding

Explain how -1 passed a bounds check despite being outside the array. Then compare that missing condition with the checks needed by the three conversion cases: where does each number stop meaning what the earlier check assumed?

Compare your reasoning

The capability is a write of a chosen int at four-byte steps from slots. Indices zero through seven and representable negative indices pass the check; indices eight and above do not. The demonstrated out-of-bounds reach is therefore before the array, and an inaccessible destination can still fault.

The two descriptions belong together: show the unauthorized store, then identify the missing lower bound that let it happen. Either the explicit two-sided check or the unsigned comparison shown above can reject that index. Other conversions need separate reasoning about the value at the point of use.

The demonstrated result is a changed flag. What a larger depot system would do with that flag remains a separate question.

Supplying an address instead

The consignment index addressed memory at a fixed step from a base you did not choose. Even with control of the stored value, it couldn’t name an arbitrary byte address.

The next lesson is the one where that stops. A format string is interpreted at run time by a function that will happily take an address from its arguments, and the reach stops being an offset from anything.