RGB to Grayscale

RGB to Grayscale

In order to convert RGB or BGR color image to grayscale image, we are frequently use the following conversion formulae:
Luminace = 0.3086 * Red + 0.6094 * Green + 0.0820 * Blue
Luminace = 0.299 * Red + 0.587 * Green + 0.114 * Blue


Notice that the luminance intensity is a sum of different weight of each color component. If we use same weight, for example, (R + G + B) / 3, then pure red, pure green and pure blue result in same gray scale level. And the other reason for using different weights is that human eye is more sensitive on green and red components than blue channel.

Fast Approximation

Implementing the conversion formula is quite simple in C/C++;

// interleaved color, RGBRGB...
for ( i = 0; i < size; i += 3 )
{
    // add weighted sum then round
    out[i] = (unsigned char)(0.299*in[i] + 0.587*in[i+1] + 0.114*in[i+2] + 0.5);
}

Since multiplication in integer domain is much faster than in float, we rewrite the formula like this for faster computation;
Luminance = (2 * Red + 5 * Green + 1 * Blue) / 8
Furthermore, we can optimize the code by substituting the multiplication and division into bit shift operators and addition. I have found this is about 4 times faster than the original float computation.

int tmp;
for ( i = 0; i < size; i += 3 )
{
    // faster computation, (2*R + 5*G + B) / 8
    tmp = in[i] << 1;                   // 2 * red
    tmp += in[i+1] << 2 + in[i+1];      // 5 * green
    tmp += in[i+2];                     // 1 * blue
    out[i] = (unsigned char)(tmp >> 3); // divide by 8
}

OpenGL Pipeline

OpenGL Pipeline has a series of processing stages in order. Two graphical information, vertex-based data and pixel-based data, are processed through the pipeline, combined together then written into the frame buffer. Notice that OpenGL can send the processed data back to your application. (See the grey colour lines)

OpenGL Pipeline

Display List

Display list is a group of OpenGL commands that have been stored (compiled) for later execution. All data, geometry (vertex) and pixel data, can be stored in a display list. It may improve performance since commands and data are cached in a display list. When OpenGL program runs on the network, you can reduce data transmission over the network by using display list. Since display lists are part of server state and reside on the server machine, the client machine needs to send commands and data only once to server's display list.
 

Vertex Operation

Each vertex and normal coordinates are transformed by GL_MODELVIEW matrix (from object coordinates to eye coordinates). Also, if lighting is enabled, the lighting calculation per vertex is performed using the transformed vertex and normal data. This lighting calculation updates new color of the vertex.
 

Primitive Assembly

After vertex operation, the primitives (point, line, and polygon) are transformed once again by projection matrix then clipped by viewing volume clipping planes; from eye coordinates to clip coordinates. After that, perspective division by w occurs and viewport transform is applied in order to map 3D scene to window space coordinates. Last thing to do in Primitive Assembly is culling test if culling is enabled.
 

Pixel Transfer Operation

After the pixels from client's memory are unpacked(read), the data are performed scaling, bias, mapping and clamping. These operations are called Pixel Transfer Operation. The transferred data are either stored in texture memory or rasterized directly to fragments.
 

Texture Memory

Texture images are loaded into texture memory to be applied onto geometric objects.
 

Raterization

Rasterization is the conversion of both geometric and pixel data into fragment. Fragments are a rectangular array containing color, depth, line width, point size and antialiasing calculations (GL_POINT_SMOOTH, GL_LINE_SMOOTH, GL_POLYGON_SMOOTH). If shading mode is GL_FILL, then the interior pixels (area) of polygon will be filled at this stage. Each fragment corresponds to a pixel in the frame buffer.
 

Fragment Operation

It is the last process to convert fragments to pixels onto frame buffer. The first process in this stage is texel generation; A texture element is generated from texture memory and it is applied to the each fragment. Then fog calculations are applied. After that, there are several fragment tests follow in order; Scissor Test ⇒ Alpha Test ⇒ Stencil Test ⇒ Depth Test.
Finally, blending, dithering, logical operation and masking by bitmask are performed and actual pixel data are stored in frame buffer.
 

Feedback

OpenGL can return most of current states and information through glGet*() and glIsEnabled() commands. Further more, you can read a rectangular area of pixel data from frame buffer using glReadPixels(), and get fully transformed vertex data using glRenderMode(GL_FEEDBACK). glCopyPixels() does not return pixel data to the specified system memory, but copy them back to the another frame buffer, for example, from front buffer to back buffer.

OpenGL Introduction

OpenGL Introduction

OpenGL is a software interface to graphics hardware. It is designed as a hardware-independent interface to be used for many different hardware platforms. OpenGL programs can also work across a network (client-server paradigm) even if the client and server are different kinds of computers. The client in OpenGL is a computer on which an OpenGL program actually executes, and the server is a computer that performs the drawings.

OpenGL uses the prefix gl for core OpenGL commands and glu for commands in OpenGL Utility Library. Similarly, OpenGL constants begin with GL_ and use all capital letters. OpenGL also uses suffix to specify the number of arguments and data type passed to a OpenGL call.

glColor3f(1, 0, 0);         // set rendering color to red with 3 floating numbers
glColor4d(0, 1, 0, 0.2);    // set color to green with 20% of opacity (double)
glVertex3fv(vertex);        // set x-y-z coordinates using pointer

State Machine

OpenGL is a state machine. Modes and attributes in OpenGL will be remained in effect until they are changed. Most state variables can be enabled or disabled with glEnable() or glDisable(). You can also check if a state is currently enabled or disabled with glIsEnabled(). You can save or restore a collection of state variables into/from attribute stacks using glPushAttrib() or glPopAttrib(). GL_ALL_ATTRIB_BITS parameter can be used to save/restore all states. The number of stacks must be at least 16 in OpenGL standard.
(Check your maximum stack size with glinfo.)
 

glPushAttrib(GL_LIGHTING_BIT);    // elegant way to change states because
    glDisable(GL_LIGHTING);       // you can restore exact previous states
    glEnable(GL_COLOR_MATERIAL);  // after calling glPopAttrib()
glPushAttrib(GL_COLOR_BUFFER_BIT);
    glDisable(GL_DITHER);
    glEnable(GL_BLEND);



glPopAttrib();                    // restore GL_COLOR_BUFFER_BIT
glPopAttrib();                    // restore GL_LIGHTING_BIT

glBegin() and glEnd()

In order to draw geometric primitives (points, lines, triangles, etc) in OpenGL, you can specify a list of vertex data between glBegin() and glEnd(). This method is called immediate mode. (You may draw geometric primitives using other methods such as vertex array.)

glBegin(GL_TRIANGLES);
    glColor3f(1, 0, 0);     // set vertex color to red
    glVertex3fv(v1);        // draw a triangle with v1, v2, v3
    glVertex3fv(v2);
    glVertex3fv(v3);
glEnd();

There are 10 types of primitives in OpenGL; GL_POINTS, GL_LINES, GL_LINE_STRIP, GL_LINE_LOOP, GL_TRIANGLES, GL_TRIANGLE_STRIP, GL_TRIANGLE_FAN, GL_QUADS, GL_QUAD_STRIP, and GL_POLYGON.

glFlush() & glFinish()

Similar to computer IO buffer, OpenGL commands are not executed immediately. All commands are stored in buffers first, including network buffers and the graphics accelerator itself, and are awaiting execution until buffers are full. For example, if an application runs over the network, it is much more efficient to send a collection of commands in a single packet than to send each command over network one at a time.
glFlush() empties all commands in these buffers and forces all pending commands will to be executed immediately without waiting buffers are full. Therefore glFlush() guarantees that all OpenGL commands made up to that point will complete executions in a finite amount time after calling glFlush(). And glFlush() does not wait until previous executions are complete and may return immediately to your program. So you are free to send more commands even though previously issued commands are not finished.
glFinish() flushes buffers and forces commands to begin execution as glFlush() does, but glFinish() blocks other OpenGL commands and waits for all execution is complete. Consequently, glFinish() does not return to your program until all previously called commands are complete. It might be used to synchronize tasks or to measure exact elapsed time that certain OpenGL commands are executed.

glVertexArray Explained

Instead you specify individual vertex data in immediate mode (between glBegin() and glEnd() pairs), you can store vertex data in a set of arrays including vertex coordinates, normals, texture coordinates and color information. And you can draw geometric primitives by dereferencing the array elements with array indices.

Take a look the following code to draw a cube with immediate mode.
Each face needs 4 times of glVertex*() calls to make a quad, for example, the quad at front is v0-v1-v2-v3. A cube has 6 faces, so the total number of glVertex*() calls is 24. If you also specify normals and colors to the corresponding vertices, the number of function calls increases to 3 times more; 24 of glColor*() and 24 of glNormal*().
The other thing that you should notice is the vertex "v0" is shared with 3 adjacent polygons; front, right and up face. In immediate mode, you have to provide the shared vertex 3 times, once for each face as shown in the code.

glBegin(GL_QUADS);      // draw a cube with 6 quads
    glVertex3fv(v0);    // front face
    glVertex3fv(v1);
    glVertex3fv(v2);
    glVertex3fv(v3);

    glVertex3fv(v0);    // right face
    glVertex3fv(v3);
    glVertex3fv(v4);
    glVertex3fv(v5);

    glVertex3fv(v0);    // up face
    glVertex3fv(v5);
    glVertex3fv(v6);
    glVertex3fv(v1);

    ...                 // draw other 3 faces

glEnd();

Using vertex arrays reduces the number of function calls and redundant usage of shared vertices. Therefore, you may increase the performance of rendering. Here, 3 different OpenGL functions are explained to use vertex arrays; glDrawArrays(), glDrawElements() and glDrawRangeElements(). Although, better approach is using vertex buffer objects (VBO) or display lists.

Initialization

OpenGL provides glEnableClientState() and glDisableClientState() functions to activate and deactivate 6 different types of arrays. Plus, there are 6 functions to specify the exact positions(addresses) of arrays, so, OpenGL can access the arrays in your application.

• glVertexPointer():  specify pointer to vertex coords array
• glNormalPointer():  specify pointer to normal array
• glColorPointer():  specify pointer to RGB color array
• glIndexPointer():  specify pointer to indexed color array
• glTexCoordPointer():  specify pointer to texture cords array
• glEdgeFlagPointer():  specify pointer to edge flag array

Each specifying function requires different parameters. Please look at OpenGL function manuals. Edge flags are used to mark whether the vertex is on the boundary edge or not. Hence, the only edges where edge flags are on will be visible if glPolygonMode() is set with GL_LINE.
Notice that vertex arrays are located in your application(system memory), which is on the client side. And, OpenGL on the server side gets access to them. That is why there are distinctive commands for vertex array; glEnableClientState() and glDisableClientState() instead of using glEnable() and glDisable().

glDrawArrays()

glDrawArrays() reads vertex data from the enabled arrays by marching straight through the array without skipping or hopping. Because glDrawArrays() does not allows hopping around the vertex arrays, you still have to repeat the shared vertices once per face.
glDrawArrays() takes 3 arguments. The first thing is the primitive type. The second parameter is the starting offset of the array. The last parameter is the number of vertices to pass to rendering pipeline of OpenGL.
For above example to draw a cube, the first parameter is GL_QUADS, the second is 0, which means starting from beginning of the array. And the last parameter is 24: a cube requires 6 faces and each face needs 4 vertices to build a quad, 6 × 4 = 24.

GLfloat vertices[] = {...}; // 24 of vertex coords
...
// activate and specify pointer to vertex array
glEnableClientState(GL_VERTEX_ARRAY);
glVertexPointer(3, GL_FLOAT, 0, vertices);

// draw a cube
glDrawArrays(GL_QUADS, 0, 24);

// deactivate vertex arrays after drawing
glDisableClientState(GL_VERTEX_ARRAY);

As a result of using glDrawArrays(), you can replace 24 glVertex*() calls with a single glDrawArrays() call. However, we still need to duplicate the shared vertices, so the number of vertices defined in the array is still 24 instead of 8. glDrawElements() is the solution to reduce the number of vertices in the array, so it allows transferring less data to OpenGL.
The size of vertex coordinates array is now 8, which is exactly same number of vertices in the cube without any redundant entries.
Note that the data type of index array is GLubyte instead of GLuint or GLushort. It should be the smallest data type that can fit maximum index number in order to reduce the size of index array, otherwise, it may cause performance drop due to the size of index array. Since the vertex array contains 8 vertices, GLubyte is enough to store all indices.

Different normals at shared vertex

Another thing you should consider is the normal vectors at the shared vertices. If the normals of the adjacent polygons at a shared vertex are all different, then normal vectors should be specified as many as the number of faces, once for each face.

For example, the vertex v0 is shared with the front, right and up face, but, the normals cannot be shared at v0. The normal of the front face is n0, the right face normal is n1 and the up face is n2. For this situation, the normal is not the same at a shared vertex, the vertex cannot be defined only once in vertex array any more. It must be defined multiple times in the array for vertex coordinates in order to match the same amount of elements in the normal array.

glDrawRangeElements()

Like glDrawElements(), glDrawRangeElements() is also good for hopping around vertex array. However, glDrawRangeElements() has two more parameters (start and end index) to specify a range of vertices to be prefetched. By adding this restriction of a range, OpenGL may be able to obtain only limited amount of vertex array data prior to rendering, and may increase performance.
The additional parameters in glDrawRangeElements() are start and end index, then OpenGL prefetches a limited amount of vertices from these values: end - start + 1. And the values in index array must lie in between start and end index. Note that not all vertices in range (start, end) must be referenced. But, if you specify a sparsely used range, it causes unnecessary process for many unused vertices in that range.

GLfloat vertices[] = {...};     // 8 of vertex coords
GLubyte indices[] = {0,1,2,3,   // 24 of indices
                     0,3,4,5,
                     0,5,6,1,
                     1,6,7,2,
                     7,4,3,2,
                     4,7,6,5};
...
// activate and specify pointer to vertex array
glEnableClientState(GL_VERTEX_ARRAY);
glVertexPointer(3, GL_FLOAT, 0, vertices);

// draw first half, range is 6 - 0 + 1 = 7 vertices
glDrawRangeElements(GL_QUADS, 0, 6, 12, GL_UNSIGNED_BYTE, indices);

// draw second half, range is 7 - 1 + 1 = 7 vertices
glDrawRangeElements(GL_QUADS, 1, 7, 12, GL_UNSIGNED_BYTE, indices+12);

// deactivate vertex arrays after drawing
glDisableClientState(GL_VERTEX_ARRAY);

You can find out maximum number of vertices to be prefetched and the maximum number of indices to be referenced by using glGetIntegerv() with GL_MAX_ELEMENTS_VERTICES and GL_MAX_ELEMENTS_INDICES.
Note that glDrawRangeElements() is available OpenGL version 1.2 or greater.