Textmode Colors

Nightshadow / Fissure & Flood

In this article I am going to explain to you how to output pictures that use RGB colors in textmode.

I assume you already know this:

 - how textmode (mode 3) works (bin style format).
 - how to handle the screen: vr-screens, palette changes in textmode, etc.

Some abbreviations I will use in this document:

 - FG = Foreground.
 - BG = Background.
 - R = Red, G = Green, B = Blue.
 - RG/GB/BR = Combinations with RGB.
 - 50%/100% = Strength of the color.
 - RGB [ X, Y, Z ] =>  X = Red, Y = Green, Z = Blue.
 - Lumin = Luminance (Strength of the color).

Okay...

- In textmode (mode 3) we've got 16 colors, 0..15.

- We have control over the characters on the screen, the BG colors and the FG colors.

- The BG colors are 0..7, the FG ones are 0..15.

- The characters which are useful for textmode demos are: the blocks (Char# 176,177,178) and the block (Char# 219).

I've found a nice way to convert RGB to textmode colors. It is based on a few simple principles:

1) Building the new palette for mode 3.

2) Converting a pixel from RGB representation to Hue & Lumin representation.

3) Converting the Hue & Lumin representation to Textmode WORD. (That WORD will be in Bin style, so it just has to be put to seg $B800.)

First let's see what this Hue & Lumin method is.

The Hue & Lumin method

The Hue determines which of the RGB components of a certain color is/are strongest. Since textmode limits us to eight BG Colors, I decided to have eight Hues. Those Hues will be: Black, White, R, G, B, RG, GB, BR. The other eight colors (8..15), which are only available for the FG, will use these Hues again, but this time, the colors will be stronger.

Since we don't need the Hue 'Black' in the BG *AND* in the FG, I dropped it from the FG and replaced it with a new color - gray. (Note: gray is NOT a Hue, it's just an additional color, and we'll use it.)

I use the following Textmode palette:

         0 - Black
         1 - White     (50% Strength, i.e. weak hue)
         2 - R         (50% Strength)
         3 - RG        (e.g. RED + GREEN at 50% --> RGB [ 32, 32, 0 ])
         4 - G
         5 - GB        (e.g. RGB [ 0, 32, 32 ])
         6 - B
         7 - BR
         8 - R         (100% Strength, i.e. strong hue)
         9 - RG        (100% Strength)
        10 - G
        11 - GB        (e.g RGB [ 0, 63, 63 ])
        12 - B
        13 - BR
        14 - Gray      (R=B=G=23)
        15 - White     (100% Strength)

Converting RGB to Hue and Lumin

The conversions "RGB -> Hue" and "RGB -> Lumin" are done separately. Let's start with the easy one: RGB -> Lumin. The easiest way to find the Lumin is:

 Lumin = Max ( R,G,B ) div Factor

Max ( R,G,B ) can be obtained easily by using a few comparisons between R, G, and B. Note: Of course your code has to be able to handle the case that two or even three RGB components are equal, too.

The Factor is different for each Bpp (bits per pixel) level. For example, in mode 13h, where 0 =< Max ( R,G,B ) =< 63, we need to set the Factor to 7, because dividing by 7 maps the values 0..63 to 0..9.

Later on you will be able to construct better RGB -> Lumin algorithms, but at the beginning let's leave it like that.

As you've noticed the Lumin is determined according to the strongest RGB component. I call it the Dominant Color.

In Lumin, the dominant color is either Red, Green, or Blue. This is not the case in the RGB -> Hue conversion. We have eight Hues. This means that we need to find the dominant color among R, G, B, RG, GB, BR, RGB, and Black.

I've built a nice algorithm that finds the dominant color. It may seem a little complex, but it is important to understand it. Note: In this algorithm I use the value '10' as the threshold.

 If R=G=B=0, Then it's (*)black.
 DeltaRG = Absolute ( R - G )
 DeltaGB = Absolute ( G - B )
 If ( DeltaRG <= 10 ) AND ( DeltaGB <= 10 ), Then it's (*)white.
 If ( R > G ) AND ( R > B ), Then it's (*)red   **EXCEPT**
  1) If ( G > B ) AND (( R - G ) <= 10 ), Then it's (*)red+green.
  2) If ( B > G ) AND (( R - B ) <= 10 ), Then it's (*)red+blue.
     [The case B=G is already covered in the (*)white calc.]
 If ( G > R ) AND ( G > B ), Then it's (*)green **EXCEPT**
  1) If ( R > B ) AND (( G - R ) <= 10 ), Then it's (*)green+red.
  2) If ( B > R ) AND (( G - B ) <= 10 ), Then it's (*)green+blue.
 If ( B > R ) AND ( B > G ), Then it's (*)blue  **EXCEPT**
  1) If ( R > G ) AND (( B - R ) <= 10 ), Then it's (*)blue+red.
  2) If ( G > R ) AND (( B - G ) <= 10 ), Then it's (*)blue+green.

The main idea here is finding what the dominant color of R, G, and B is, and after finding it, checking if the second highest RGB component is close enough to the dominant color (as I said, the value 10 is the threshold). If the answer is yes, the resulting color will be a blend of these two colors; if not, it will be the dominant color alone. For example:

 RGB [ 10,13,19 ] -> White Hue. (Distance between all RGB components < 10.)
 RGB [ 10,10,21 ] -> Blue. (Distance > 10.)
 RGB [ 10,11,21 ] -> Blue+Green. (Distance between B and G < 10. Of course if
                     the threshold was less than 10, e.g. 9, RGB [ 10,11,21 ]
                     would be converted to Blue.)
 RGB [ 11,10,21 ] -> Blue+Red.

Notice that in our HUE representation of R/G/B/RG/GB/BR/Black/White, we don't consider what the dominant color and what the second dominant color is. If there are two dominant colors (i.e. the delta between the first dominant and the second dominant color is less or equal to 10), we DON'T distinguish between them.

In other words, if our Hue is RG, there can be two reasons for it: either R was the dominant color and G the second dominant, or vice versa. In both cases we get the same Hue, and this means that later on it will become the same "text pixel" (i.e. character). (Maybe the lumin might change it a bit, but the FG and BG colors will definitely be the same. Just continue reading, and you'll see why.)

Converting Hue & Lumin to WORD

Now that we've converted RGB to Hue & Lumin, it's much easier to choose what text pixel we need to output.

Here's the algorithm for the Hue & Lumin -> Word conversion. I advise you to implement this as a function, not a procedure.

[General cases: black hue, black lumin or gray lumin]

 If Hue = Black, Then textpixel = black text pixel. Exit procedure.
    [Black text pixel: FG+BG colors = 0 with a space char ' ']
 If Lumin = 0, Then textpixel = black text pixel. Exit procedure.
 If Lumin = 1, Then textpixel = gray color = FG col #14, BG col = 0,
                    '' block.
                    Exit procedure.

[Normal cases: the hue is not black, and the lumin > 1]

 If Hue = White, Then let's check the Lumin:
  If Lumin = 2 Then '' +  FG = 1   +  BG = 0 [col  #1 =  50% white]
             3 Then '' +  FG = 1   +  BG = 0 [col #15 = 100% white]
             4 Then '' +  FG = 1   +  BG = 0
             5 Then '' +  FG = 1   +  BG = 0
             6 Then '' +  FG = 15  +  BG = 1
             7 Then '' +  FG = 15  +  BG = 1
             8 Then '' +  FG = 15  +  BG = 1
             9 Then '' +  FG = 15  +  BG = 1

Well, these last lines contained the main idea of textmode colors. By now you should know why we have ten lumin levels for each hue. I assume you've understood how to complete the algorithm. Yet I'll supply two more cases which are similar to Hue = White. The only thing that is different is the numbers of the FG/BG colors, but the principle remains the same.

 If Hue = Red, Then let's check the lumin:
  If Lumin = 2 Then '' +  FG = 2   +  BG = 0 [col #2 =  50% R]
             3 Then '' +  FG = 2   +  BG = 0 [col #8 = 100% R]
             4 Then '' +  FG = 2   +  BG = 0
             5 Then '' +  FG = 2   +  BG = 0
             6 Then '' +  FG = 8   +  BG = 2
             7 Then '' +  FG = 8   +  BG = 2
             8 Then '' +  FG = 8   +  BG = 2
             9 Then '' +  FG = 8   +  BG = 2

 If Hue = Red+Green, Then let's check the lumin:
  If Lumin = 2 Then '' +  FG = 3   +  BG = 0 [col #3 =  50% RG]
             3 Then '' +  FG = 3   +  BG = 0 [col #9 = 100% RG]
             4 Then '' +  FG = 3   +  BG = 0
             5 Then '' +  FG = 3   +  BG = 0
             6 Then '' +  FG = 9   +  BG = 9
             7 Then '' +  FG = 9   +  BG = 9
             8 Then '' +  FG = 9   +  BG = 9
             9 Then '' +  FG = 9   +  BG = 9

Optimizing this algorithm

As you've seen the textpixel character () can be determined quickly, outside the inner loop of each color, but the FG/BG colors must be determined in the inner loop because they are different for every Hue. The only 'If' you'll have inside each Hue, will be about its lumin: 2..5 or 6..9.

And even the new optimized code can run faster, but I see no point in such an optimization, because after writing this Hue & Lumin -> Word procedure, I made a lookup table of all possible Hue & Lumin combinations and their matching Words.

Just in case you don't want to have to find out by yourself how the "bin" format of the textmode colors works, here are important parts of my conversion routines.

This routine shows where the BG and FG colors 'sit' in their byte.

 Function ColorBin ( BG, FG : Byte ) : Byte ;
  Begin
   ColorBin := ( ( BG Shl 4 ) + FG  ) ;
  End ;

And somewhere in the middle of my HueLumin2Word function there are these lines...

 ...
 If Lumin = 1 Then { = gray  }
  Begin
   Char  := 176 ;
   Color := ColorBin ( 0, 14 ) ; { ColorBin ( BG, FG ) }
 ...
 If ( Lumin = 2 ) Or ( Lumin = 6 ) Then Char := 176 ; {  }
 If ( Lumin = 3 ) Or ( Lumin = 7 ) Then Char := 177 ; {  }
 If ( Lumin = 4 ) Or ( Lumin = 8 ) Then Char := 178 ; {  }
 If ( Lumin = 5 ) Or ( Lumin = 9 ) Then Char := 219 ; {  }
 ...

So the integer 'char' holds the char byte, and 'color' is the color byte. At the end of these functions they are united to the desired Word:

 BinResult := Char + Color Shl 8 ; { Char / Color }

Adjoined with this document, there's an .exe file I've written, which actually displays a fire in mode13h, in a window of 80x50. I converted it with my routines to textmode. One of the mouse buttons changes the pointer from an arrow to an omni, and the other button increases the lumin for the whole screen. BTW, in this program I use a lookup table of 256 words which converts any color (0..255) of mode 13h to a Word (.Bin style).

Closing Words

This document is part of a set of documents on 2D effects (Soften, Zoomer, Tunnel, Water, etc.) which I am currently writing. They are primarily designed for newbies but will also contain optimization hints as well as simple source codes. I'd be very happy if anyone helped me write this tutorial set.

If you've found this document helpful or have any suggestions, I'd be very happy to hear 'em from ya. My mailbox at ShadowCraft_@hotmail.com is waiting to collect comments.

- NightshadoW / Fissure & Flood 1999. (Israel)