NES Colour: Method

Display hardware

  • Display hardware varies over test runs, however comparisons are only done on the same display (i.e.: I'll never compare a test run on a Sony PVM to domestic display, or between two different PVMs).
  • Make and model will be noted on each comparison.
  • Scan converters and line doublers will be noted if in use, and what settings are used that may affect colour or brightness.
  • CRT displays will be calibrated to Rec.601 / ITU BT.601
    • Peak luminance of 100nits for NES peak white
    • D65 white point
  • LCD/OLED displays will be calibrated to Rec.709 / ITU BT.1886
    • Gamma 2.4
    • Peak luminance of 100 nits for NES peak white
    • D65 white point
  • Calibrated colour values will not change between tests
  • Black and white level will be adjusted to match NES colours 00 (dark grey) and 20/30 (peak white) before each palette data capture

NES luma

  • According to the NES DevWiki NES peak luma is slightly above broadcast max (110 IRE instead of 100). Levels to use as a rough target:
    • 00 is 43 IRE, adjusted to 39.1
    • 10 is 74 IRE, adjusted to 67.3
    • 20/30 are 110 IRE, adjusted to 100.0
  • If one of the comparison systems + display can't reach 100 nits, the maximum value will be noted and used as the new peak.

Colorimeter

  • Unless otherwise noted, the colorimeter used is a ColorMunki Display, model number CMUNDIS.
    • Identical hardware to i1 Display Pro / iD3.

Software

  • I always try to use Free and Open Source Software where possible (and thanks to every single author/developer/tester/documenter/researcher who made the following free and open!)
  • Displays are calibrated with HCFR.
  • Colour measurements are taken with ArgyllCMS spotread.
  • 240p Test Suite NES version is used for colour patch generation, as well as PLUGE and peak brightness levels.
  • "colormath" Python libraries are used for colour conversions
  • "numpy" Python libraries are used for statistics
  • Scripts and tools to generate data on GitHub: https://github.com/danmons/nescolour
  • I run all this in Linux (with the exception of HCFR which is Windows-only). It'll probably work on Windows inside Cygwin, WSL or WSL2, but is untested.

Video game systems

  • I currently have access to:
    • Japanese Nintendo AV Famicom model HVC-101 with native composite out
    • Australian Nintendo NES model NESE-001 with native composite out
    • MiSTer FPGA with IO board and direct video (colour system and connector type used will be noted per test)
    • Raspberry Pi 2/3/4 (colour system and connector type used will be noted per test)
  • Specific comparisons will note which of these I'm comparing.
  • Currently the Japanese AV Famicom is considered the default reference point, until I can get my hands on either a Japanese or US NES-001 (if you can assist me in finding one that can ship to Australia for a reasonable price, please contact me)
  • For now I'm focusing on MiSTer FPGA and palettes as the comparison point. Emulators on Raspberry Pi may come later.

Process

  • Allow display to warm up for a minimum of 15 minutes
  • Attach colorimeter to laptop, ensure it's working
  • Calibrate display to standards mentioned above in "Display hardware" via colorimeter and HCFR
  • Attach reference video game system to display
    • Run 240p Test Suite
    • Adjust black level via PLUGE
    • Measure white (colour 20/30) level, ensure 100nits peak or take note if lower
    • Measure dark grey (colour 00) level, take note of value
  • Measure colour patches with ArgyllCMS spotread
    • Use the following command on the laptop: spotread -c1 -yr -ew | tee outputfile1.txt
    • Set 240p Test Suite to pure white
    • Take first reference measurement of pure white to compensate for white balance
    • Set 240p Test Suite to custom colour "00"
    • Press "Enter" on laptop to take a measurement
    • Repeat for custom colours "01" through to "3C"
    • Press "q" on laptop twice to quit spotread
  • Collect data
    • Run: grep -i result outputfile1.txt > result1.txt
  • Attach comparison video game system to display
    • Select palette to be investigated
    • Run 240p Test Suite
    • Adjust white point (colour 20/30) to match reference video game system we noted earlier
    • Adjust dark grey (colour 00) to match reference video game system we noted earlier
    • Repeat measurement and collection steps to new files

Interpret results

  • Calculate dE values with script "compare.sh result1.txt result2.txt > compare_1_2.txt"
  • Generate statistics with script "stats.sh compare_1_2.txt"
    • dE mean (average value)
    • dE standard deviation (how variable the results are across all colours)
    • 10th percentile dE (indicates best value, ignores outliers and pure white)
    • 90th percentile dE (indicates worst value, ignores outliers)
    • dE maximum (true worst value)
  • Lower values are better in all cases
    • dE 2.00 or lower is considered "not noticeable without close inspection", and our target maximum
    • dE 1.00 or lower is considered undetectable by humans
  • Render image comparison with "render.sh result1.txt result2.txt", which will produce render_result1_result2.png
  • Rendered image prints the colours of the two systems side by side for comparison:
    • Starts with 00 at the top left, continuing 01, 02 ... 0A
    • Second line 10 ... 1C
    • Third line 20 ... 2C
    • Fourth line 30 ... 3C
    • Left to right sweeps across hues
    • Top to bottom increases in brightness
  • Rendered image is then repeated 4 times on different backgrounds to assist with critical decision making relative to luminance:
    • Black
    • 50% grey RGB(128,128,128)
    • Middle grey RGB(188,188,188), which is 50% brightness at sRGB/Rec.601/Rec.709 on a standard Gamma 2.2 curve
    • White

Compare to your setup

  • If you're using an emulator, take a screenshot and use a colour picker, or just compare the images side by side.
  • If you're using a console, get the renders up on a screen that you can put next to your video game system (phone, tablet or laptop works well)
  • If you have the equipment (i.e.: a colorimeter or spectrometer)
    • For best results calibrate your display and computer to Rec601 (SD CRT) / Rec709 (HD LCD/OLED / sRGB (PC monitor, CRT or LCD), and match peak brightness on each
  • If you don't have the equipment
    • Attempt to eye-match PLUGE and monochrome colours 00, 10 and 20 for brightness and contrast first - avoid bloom or "blow out" in light values
    • Attempt to eye-match monochrome colours 00, 10 and 20 for colour temperature (how "warm" (red) or "cool" (blue) the greys and whites appear). Not all displays have these settings. Some laptop video card software allows adjustment, likewise for some phones and tablets (check "eye strain relief" or "blue filter" options), and most decent TVs have some sort of calibration options, even if they're presets
  • Reduce background light as much as possible (a dark room is best, especially avoid coloured lights)
  • Place screens next to each other, judge for yourself.

References

Colour Terminated Potential mV IRE PPU Absolute Voltage V PPU Relative Voltage V Normalised grey level
00 616 43 1.875 1.09 0.397
10 840 74 2.287 1.50 0.681
20 1100 110 2.742 1.96 1.000

  • Gamma https://en.wikipedia.org/wiki/Gamma_correction

  • Using custom Gamma 1.8 (coincidentally what old Macintosh machines from 1985 would use)

    • 00 at 39.7 is Grey101 approx 19% luminance
    • 10 at 68.1 is Grey173 approx 50% luminance (aka middle grey)
    • 20 at 100 is Grey255 100% luminance (white)

  • Using sRGB/NTSC/YIQ gamma 2.2

    • 00 at 39.7 is Grey101 approx 13% luminance
    • 10 at 68.1 is Grey174 approx 43% luminance
    • 20 at 100 is Grey255 100% luminance (white)

  • Using Rec.601/Rec.709+BT.1886 gamma 2.4

    • 00 at 39.7 is Grey102 approx 11% luminance
    • 10 at 68.1 is Grey174 approx 40% luminance
    • 20 at 100 is Grey255 100% luminance (white)

  • sRGB Gamma 2.2 table: