I can understand the confusion, when normally one expects, for RGB a three (or four) byte number 0xRRGGBB, when RR, GG and BB range from 0x00 to 0xFF. You may think that RGB8 is a similar standard with just 2 bits per Red, Green and Blue (and a 2 bit alpha channel). But no¹...
From the sounds of it, the image is a bitmap² and is using an 8 bit colour map, or 8bpp, so there are 256 distinct colours supported.
Each byte represents one pixel³ and is an index to a table containing 256 colours. This table is held in memory, and may contain either user defined or a standard system defined set of colours, known as a colour palette.
The actual colours in this colour palette are not set in stone. There is no actual standard, apart from a standard relating to a particular OS. It is possible to change the colours in the colour palette, so that the colours to which the indices refer change, so called colour palette animation and can be used to create some rather psychedelic effects.
The values can indeed represent an increasing greyscale, but in this case the OP states that each number represents a colour and not a shade of grey. In which case, it is using a colour palette.
So, in short, an answer to the OP's question is that there is no set colour for each pixel, the C# program merely has to obtain the colour index for each pixel. The actual colour that the index represents is irrelevant, until the colour palette has been defined.
I hope that makes sense.
As an historical aside, not so long ago (mid 80s), 8 bit video cards were once at the top end of the technology, current at that time. Then along came 16 bit colour (thousands of colours) in the early 90s, then 24 bit colour (millions of colours), in the mid 90s (Apple Quadra Series) and now 32 bit is de rigeur.
¹ Although there are exceptions (3 bits for R and G and 2 bits for B), credit goes to joojaa for the tip.
² There are a number of bitmap formats: BMP (Windows); XPM (X Windows); TIFF and; PPM to name just a few. See Bitmap image file extension list.
³ It might be worth also having a look at the Wikipedia entry for Pixel Format.