A pixel art checker for NES, SNES, SMS and Mega Drive limits

Pixel art that looks fine in an image viewer can still be impossible on the console it was drawn for. To check it, I built pixcheck, a Python script that tests a PNG against the color, palette and tile limits of the NES, Master System, SNES or Mega Drive and marks every place that breaks one. Then I asked three Claude models for 72 pieces of pixel art as code. When the prompt only named the console, 11 of 36 images broke a limit: 6 of 9 on the NES and 5 of 9 on the SNES, while all 18 Master System and Mega Drive images fit. With one extra sentence asking to respect the limits, 1 of 36 broke one.

pixcheck takes a fraction of a second on most images. For the Mega Drive it also writes an indexed PNG that SGDK’s rescomp accepts, and I checked that end to end: all 12 backgrounds came out of an emulator identical to the source, pixel for pixel.

This is the third post in the retro series, after mdprof, the SGDK profiler and sgdklint, the SGDK performance lint.

What limits does pixel art have to fit on each console?

None of these consoles draws a bitmap. A background is a grid of 8x8 tiles, each tile picks one small palette, and the number of palettes, colors and distinct tiles is fixed by the hardware. These are the limits pixcheck checks:

	NES	Master System	SNES (mode 1, 4bpp layer)	Mega Drive
Color depth	fixed hardware palette	2 bits per channel (64 colors)	5 bits (32,768)	3 bits (512)
One palette covers	a 16x16 area	an 8x8 tile	an 8x8 tile	an 8x8 tile
Colors per palette	3 + one shared backdrop	16	15 + backdrop	15 + backdrop
Background palettes	4	2	8	4
Unique 8x8 tiles	256, no flipping	448	1,024 per layer	1,425 with SGDK’s defaults
Sprite colors	3 + transparent per 8x8 tile	15 + transparent	15 + transparent	15 + transparent
Sprites on one line	8	8	32	20

The NES row is the one that catches people. Its palettes are chosen per 16x16 area, not per tile, through the attribute table: two tiles that each use three colors are fine alone and illegal side by side if they need different palettes inside the same 16x16 square. The Mega Drive budget is not the 2,048 tiles that fit in VRAM: I read SGDK 2.09’s TILE_USER_LENGTH from a running ROM, and with its default plane layout it is 1,425.

Does AI pixel art fit NES and SNES limits?

I asked Claude Sonnet 5.5, Opus 5.5 and Haiku 5.5 for three pictures per console: a sunset over mountains and a lake (full screen), a top-down dungeon room (full screen), and a 4-frame knight walk cycle as a sprite sheet (16x16 frames on the NES and Master System, 32x32 on the others). Each request went through claude -p with no tools, and the model replied with a Pillow script that I ran in a container with no network. There were two prompt variants: one that only named the console and the size, and one that added “It must work as-is on real hardware: respect the console’s color, palette and tile limits.”

Prompt	Model	First try fits	Broke a limit on
Names the console	Sonnet 5.5	7 of 12	NES 3, SNES 2
	Opus 5.5	10 of 12	SNES 2
	Haiku 5.5	8 of 12	NES 3, SNES 1
Asks for the limits	Sonnet 5.5	12 of 12
	Opus 5.5	12 of 12
	Haiku 5.5	11 of 12	NES 1

The failures were the subtle limits, not the obvious ones:

• NES attribute areas. All four failing NES backgrounds had 16x16 areas with more than 3 colors plus the backdrop (28 to 57 areas of 240), and three of them also had more than 256 unique tiles (271, 511 and 653).
• NES sprites. Both failing knights used 5 or 6 colors in every 8x8 tile. An NES sprite tile has three.
• SNES palette packing. Three of the five SNES failures used only 60 to 78 colors, well under the 120 that 8 palettes of 15 can hold, and still could not be packed: their tiles mix colors from too many different groups. pixcheck’s exact search proved there is no packing for those three.
• The one failure with the limits sentence was Haiku’s NES sunset: legal palettes, but 258 unique tiles for a budget of 256, because of dithered gradients.

The models that passed mostly did it by using little color. With the limits sentence, 24 of the 36 images used a single palette, including 7 of the 9 Mega Drive images, which had four available. The Mega Drive and Master System passes in the naive variant came the same way: 8 to 32 colors in total. That is legal, and it is also why “the AI got it right” means less than it sounds.

Here is Sonnet’s naive NES sunset. Red boxes are 16x16 areas with too many colors; orange boxes are areas whose colors fit but that got no palette, because 18 colors do not fit in four palettes of three. On the right is the version Sonnet wrote after reading pixcheck’s report:

Why won’t rescomp accept my PNG?

On the Mega Drive, SGDK’s resource compiler (rescomp) turns PNGs into tiles and palettes, so I dropped each of the 12 Mega Drive backgrounds into res/ as the models saved them. 4 built. They were indexed PNGs with 15 or 16 colors, one palette’s worth. The other 8 were RGB PNGs and stopped at the first step:

Error: expected a plain tile for palette data at position [0,0] (see rescomp.txt for more info about RGB image) !

rescomp’s documentation explains why: for an RGB image it expects the four palettes drawn as rows of 8x8 color tiles in the top-left corner, and for an indexed image it uses the indexes as they are, with bits 4 and 5 picking the palette. rescomp never packs palettes for you. Converting a legal 61-color test image to a 64-color indexed PNG with Pillow’s quantizer fails as well (pixel at [5,0] reference a different palette (2 != 3)), because a generic quantizer does not group colors by tile.

pixcheck md image.png --export res/bg.png does that grouping. It writes an indexed PNG where each tile’s pixels point into the palette pixcheck packed for it, with the backdrop as color 0 of every palette. To check that the export is right, not just accepted, I built a small SGDK ROM that shows the image and ran it headless in Genesis Plus GX through mdprof, taking a screenshot at frame 140. All 12 Mega Drive backgrounds, up to 32 colors in 4 palettes and 1,104 unique tiles, built and matched the source on all 71,680 pixels.

The first round trip I ran said 3 of 6 images differed on almost every pixel. The screenshot was taken at frame 30, while the larger images were still being copied to VRAM (11.6 KB of DMA in that frame). Taking it later fixed it.

How the checker works

For a background, pixcheck snaps every color to the console’s color grid, cuts the image into palette areas (16x16 on the NES, 8x8 tiles elsewhere) and collects the set of colors each area uses. Two things can then go wrong: an area has too many colors, or the areas’ color sets cannot be packed into the palettes the console has. The second one is a bin packing problem, and finding the minimum is NP-hard, so pixcheck tries a greedy packing first:

for s in order:                      # color sets, biggest first
    if any(s <= p for p in pals):    # already covered by a palette
        continue
    fit = [p for p in pals if len(p | s) <= capacity]
    if fit:                          # join the palette it shares the most colors with
        p = max(fit, key=lambda p: (len(p & s), -len(p)))
        p |= s
    elif len(pals) < count:
        pals.append(set(s))
    else:
        left.append(s)               # no palette for this area

The greedy packing gave me a false alarm on the real test. Opus’s SNES sunset (46 colors) left 8 tiles without a palette even after 300 shuffled restarts, but a backtracking search over the 96 maximal color sets found a packing that fills all 8 palettes with exactly 15 colors (the fix rounds Opus ran to chase that false alarm are left out of the tables here). So now, when the greedy packing leaves sets out, pixcheck runs that exact search (with a bound that stops when the colors still to place cannot fit in the free slots) and says which it was: found, proven impossible, or out of time after 120 seconds. That image takes 16.8 seconds; the median over all 72 images is 0.02 seconds.

The second mistake was NES color. There is no single true NES palette; emulators and TVs show the same index differently. My first version decoded one from the NTSC signal levels and flagged colors in every NES image as “not an NES color”. They were standard values: Opus labeled (248, 56, 0) as $16 red, the value many emulator palettes use, and my decoded $16 was a much duller red. pixcheck now takes NES colors as given, and snaps to a palette only if you pass an emulator .pal file.

Other checks: tiles are deduplicated with flips where the console can flip a background tile (not on the NES), source colors that collapse into one console color get a warning, and for sprite sheets it reports how many 8-pixel-wide sprites the busiest line of a frame needs. The synthetic tests in the repo cover each limit with images that break it a known number of times.

Does feeding the checker’s report back to the model fix it?

For the 12 failing images I tried two fixes, each starting from the first script. One said only “the image this script draws breaks the console’s hardware limits, fix it”. The other pasted pixcheck’s report, with a second round if the first did not pass.

Fix	Passed	Median change from the first image
Plain “it breaks the limits”	11 of 12	5.3 (mean RGB difference, out of 255)
pixcheck’s report	12 of 12 (one needed a second round)	7.7

The report did not make better fixes. The plain prompt was nearly as good and changed the pictures slightly less. The one plain failure was Haiku’s NES dungeon with 351 colors, which came back with 19. This matches what I found with sgdklint: once a model is told something is wrong, it mostly knows the fix.

Where the checker earns its place is the step before that: knowing something is wrong. A vibe coder cannot see an attribute clash or a 258th tile in an image viewer, and the plain prompt only works if you already know the image is broken. The other place is the Mega Drive export, which no prompt replaced: 8 of 12 model images did not get through rescomp as saved.

What broke and what I would do differently

• Only the Mega Drive went through a real toolchain and emulator. The NES, Master System and SNES results are pixcheck’s rules, checked against synthetic images, not against a ROM. NES export (CHR and attribute data) and SNES export are the obvious next steps.
• I could not cross-check the packing with another converter. I tried to build superfamiconv, which packs SNES, Master System and Mega Drive palettes, and the build of outside code was not allowed in this run. The exact search is the cross-check instead.
• Small sample. One run per model, task and console, so 12 images per model and variant. The pattern (NES and SNES fail, Master System and Mega Drive pass) is clear; the differences between models are not.
• Legal is not good. pixcheck says whether art fits, not whether it is good pixel art. Most passing images got there by using one palette. I cannot judge pixel art like an artist and did not try.

Run it

python3 pixcheck/pixcheck.py nes title.png --overlay title-check.png   # background, with the boxes drawn
python3 pixcheck/pixcheck.py md hero.png --sprite 32x32                # sprite sheet of 32x32 frames
python3 pixcheck/pixcheck.py md stage.png --export res/stage.png       # indexed PNG for SGDK rescomp

It needs Python 3 with Pillow and numpy, and exits with 1 when a limit breaks, so it can run before every build. The project folder has the checker, the tests, every script the models wrote with the image it drew and pixcheck’s report, and the Mega Drive round trip with its screenshots.