cs 354 introduction
DESCRIPTION
January 17, 2012 lecture http://www.cs.utexas.edu/~mjk/teaching/cs354_s12/TRANSCRIPT
CS 354 2
CS 354—Computer Graphics
Instructor: Mark Kilgard Principal Software Engineer, NVIDIA Previously at Silicon Graphics
Teaching Assistant: Randall Smith Ph.D. student with Dr. Fussell TA’ed the course last semester
Burdine Hall (BUR) 116 Tuesday/Thursday, 9:30-11:00 a.m.
CS 354 3
Expectations For us
Teach practical graphics expertise
Well-prepared lectures Grades fairly reflect
your participation and performance
Available to you Use my office hours Other times, use Randy If not satisfied, contact me
For you Attendance at every
class Expect regular quizzes at
the beginning of class
Programming assignments You need to know C/C+
+ Use office hours if you
need help No cheating
CS 354 4
Grading
Testing 60% Daily quizzes 10%
2-3 questions, easy
Homework 10% Occasional, mostly
math questions
Exams 40% Mid-term 15% Final 25%
Software projects 40% Simple Rendering Object loader Interaction Shading Performance analysis
CS 354 5
Textbook
Interactive Computer Graphics: A Top-Down Approach With Shader-Based OpenGL by Edward Angel and Dave Shreiner Addison-Wesley, 6th edition
Regular readings assigned Match up with lecture topics
CS 354 6
Helpful Resources Learning OpenGL
OpenGL-oriented books
Supplemental books
OpenGLProgramming
Guidea.k.a. “the red book”
OpenGLSuperBible
OpenGLA Primer
Eric Lengyel’sMathematics for 3D
Game Programmingand Computer Graphics
Real-Time Rendering byEric Haines, Tomas Akenine-Moller,Eric Haines, Naty Hoffman
CS 354 7
Computer Graphics Nexus of several disciplines
HumanPerception
ArtisticExpression
Physicsof Light
Geometry andMathematics of
Surfaces
Computer ScienceVLSI Hardware
Design Display & InputTechnology
Animation &Simulation
CS 354 8
Roles for Computer Graphics
[Pixar 2010]
Story telling
Product design
[CATIA]
Roles for Computer Graphics
CS 354 12
What does computer graphics study?
Computer-based simulation of Shape
Appearance
Motion
[Litke et.al. 2001]
[Chai & Hodgins, 2005]
[Sloan et.al. 2005]
CS 354 13
Not covered in the class
Digital content creation No Photoshop, no Maya or 3D Studio Max Computer Science class, not an art class
C/C++ programming I expect you know C or C++ under Linux
Not just the language Need to know debugging and software practices
Since course’s programming projects assume Linux software development
Use the Computer Science labs (ESN or Painter)
CS 354 14
Visual Thinking Human visual system = highest bandwidth input to your brain
Very natural to want computers “feeding” this input to your brain Because people think visually
Innate intuition for 3D imagery in particular Computer graphics
Takes an abstract representation of a “scene” within a computer’s memory and converts it to a concrete signal (an image, or animating images) representing a view of that scene
Computer graphics practitioners have a good practical and theoretical understanding of how to do this
Amazing progress in the last 40 years Your brain
Takes concrete signals (images, the visual world) and converts those signals (back!) into an abstract representation of a scene
We have only the most rudimentary notions of how this process works
Teaching a computer to do this is the field of “computer vision”
CS 354 15
Reductionist Approach Can’t simply “take a picture” like a camera
Instead images are “synthesized” from an abstract model of a scene and its view
Must build graphics out of little bits of work and data Philosophers and scientists break down phenomenon to
smallest observable units Examples
Philosophical atoms (Leucippus, ancient) Triangles (Plato, ancient) Chemical atoms (Dalton, pre-modern) Physical atoms (Bohr, modern) Sub-atomic quantum particles, waves, strings
Reductionism in graphics Pixels, vertexes, and triangles (Plato redux?) Discretization pervades, ideal for digital computers
CS 354 16
Your Modern World View:Discrete Information, Particularly Images You believe in digital information
Impossible to avoid this belief in a modern, computerized world
Text is digital, video is digital, music is digital, communication is digital, even identity is becoming digital
You believe exact copies are possible, even common Pervasive new belief that’s incognizant to world just 50
years ago Discrete images implies pixels
Descartes’s Cartesian plane computerized Image ≈ grid of discrete color So images are simply numbers
So can be processed as numbers!
CS 354 17
Shape: Objects to Triangles
Digital artists approximate solid object shape with meshes of triangles Was Plato right?
More triangles makes the mesh an increasinglyaccurate approximation of the bunny shape
CS 354 18
What will you learn
Fundamentals of computer graphics Transformations and viewing Rasterization and ray tracing Lighting and shading Graphics hardware technology Mathematics for computer graphics
Practical graphics programming OpenGL programming Shader programming Performance analysis
CS 354 19
Expect programming projects using OpenGL Explicitly not just an “OpenGL” class
But you will learn and use OpenGL When relevant Direct3D is discussed too
You’ll use GLUT so programs can work on Windows, Mac, and Linux
Plenty of resources to learn OpenGL www.opengl.org
CS 354 20
What is OpenGL? Its specification is titled “The OpenGL Graphics
System” Not just for 3D graphics; imaging too “GL” standard for “Graphics Library” “Open” means industry standard meant for broad
adoption with liberal licensing Standardized in 1992
By Silicon Graphics And others: Compaq, DEC, Intel, IBM, Microsoft Originally meant for Unix and Windows workstations
Now de facto graphics acceleration standard Now managed by the Khronos industry consortium Available everywhere, from supercomputers to cell
phones Alternative: Direct3D provides similar functionality
with a very different API for Microsoft Windows platforms
[Marathon Oil visionarium]
[ES2 tablet]
[MacBook]
CS 354 21
OpenGL as an Evolving Standard
29%
17%
15%
15%4%
2%
2%
2%2%
2%
2%
2%
1% 1%
4%
15%
Multi-vendor
Silicon Graphics
Architectural Review Board
NVIDIA
ATI
Apple
Mesa3D
Sun Microsystems
OpenGL ES
OpenML
IBM
Intense3D
Hewlett Packard
3Dfx
Other
EXT
SGISGISSGIX
ARB
NV
Others Others
44% of extensions are “core” or multi-vendor Lots of vendors have initiated extensions
Extending OpenGL is industry-wide collaboration
ATI
APPLE
MESA
Source: http://www.opengl.org/registry (Dec 2008)
CS 354 22
Forces Driving Improvements in Computer Graphics
Human desire for VisualIntuition and Entertainment
Embarrassing
Parallelism ofGraphics
Increasing
Semiconductor
Density
Particularly thehardware-amenable,
latency tolerantnature of rasterization
Particularlyinteractive video games
ComputerGraphics
Moore’sLaw
CS 354 23
Key Trend in OpenGL Evolution
Fixed-function Programmable
SimpleConfigurability
ComplexConfigurability
Shaders!
High-level languages
CS 354 24
Many Perspectives on OpenGL
Programmer’s view: Application Programming Interface (API) Accelerated access to 3D graphics hardware
Performance and functionality improves with time Graphics Architect’s view:
Detailed functional graphics pipeline Well suited to VLSI hardware design
Software or Hardware System Designer’s view: Stable standard component for building larger system
Platform neutral so fits into other systems well Examples: WebGL, Apple’s compositing desktop
Student’s view: Real-world implementation of graphics concepts
Good pedantic structure for learning graphics Practical skill you can really use
CS 354 25
Programmer’s View:OpenGL API Example
Let’s draw a triangleglShadeModel(GL_SMOOTH); // smooth color interpolationglEnable(GL_DEPTH_TEST); // enable hidden surface removal
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT); glBegin(GL_TRIANGLES); { // every 3 vertexes makes a triangle glColor4ub(255, 0, 0, 255); // RGBA=(1,0,0,100%) glVertex3f(-0.8, 0.8, 0.3); // XYZ=(-8/10,8/10,3/10)
glColor4ub(0, 255, 0, 255); // RGBA=(0,1,0,100%) glVertex3f( 0.8, 0.8, -0.2); // XYZ=(8/10,8/10,-2/10)
glColor4ub(0, 0, 255, 255); // RGBA=(0,0,1,100%) glVertex3f( 0.0, -0.8, -0.2); // XYZ=(0,-8/10,-2/10)} glEnd();
Pro Tip: use curly braces to “bracket” nested OpenGLusage; no semantic meaning, just highlights grouping
CS 354 26
Initial Logical Coordinate System
Think of drawing into a [-1,+1]3 cube
(-0.8, 0.8) (-0.8, 0.8)
(0, -0.8)
origin at (0,0)
CS 354 27
Visualizing Normalized Device Coordinates
What does this simple triangle look like with the[-1,+1]3 cube’s coordinate system? We call this coordinate system “Normalize Device Coordinate”
or NDC spaceWire frame cube showsboundaries of NDC space
From NDC views, youcan see triangle isn’t“flat” in the Z direction
Two vertices have Z of -0.2—third has Z of 0.3
CS 354 28
Programmer’s View:GLUT API Example
Windowing code#include <GL/glut.h> // includes necessary OpenGL headers
void display() { // << insert code on prior slide here >> glutSwapBuffers();}
void main(int argc, char **argv) { // request double-buffered color window with depth buffer glutInitDisplayMode(GLUT_RGBA | GLUT_DOUBLE | GLUT_DEPTH); glutInit(&argc, argv); glutCreateWindow(“simple triangle”); glutDisplayFunc(display); // function to render window glutMainLoop();}
FYI: GLUT = OpenGL Utility Toolkit
CS 354 29
Reductionism for Rendering Objects and Scenes
Programmers want to render “objects” Say a fire truck or molecule Arranged relative to other objects (a scene) & then viewed
Graphics pipeline approach—used by OpenGL and GPUs Break objects into geometry batches
Batches may be meshes or “patches” Batches reduce to polygonal primitives
Typically triangles But also lines, points, bitmaps, or images
Geometric primitives are specified by vertices So vertices are assembled into primitives
Primitives are rasterized into fragments Fragments are shaded Raster operations take shaded fragments and
update the framebuffer
CS 354 30
Programmer’s View: AlsoProgramming Shaders inside GPU
Multiple programmable domains within the GPU Can be programmed in high-level language
OpenGL Shading Language (GLSL)
GeometryProgram
3D Applicationor Game
OpenGL API
GPUFront End
VertexAssembly
VertexShader
Clipping, Setup,and Rasterization
FragmentShader
Texture Fetch
RasterOperations
Framebuffer Access
Memory Interface
CPU – GPU Boundary
OpenGL 3.3
Attribute Fetch
PrimitiveAssembly
Parameter Buffer Readprogrammable
fixed-function
Legend
CS 354 31
Example Simple GLSL Shaders
Vertex Shader Operates on each vertex
of geometric primitives
Passes through per-vertex color
Transforms the vertex to match fixed-function processing
Fragment Shader Operates on each
fragment (think pixel)
Outputs the fragment’s interpolated color to the framebuffer
void main(void){ gl_FrontColor = gl_Color; gl_Position = ftransform();}
void main(void){ gl_FragColor = gl_Color;}
Shaders are way more interesting than these minimal examples
CS 354 34
OpenGL’s Design Philosophy
High-performance Assumes hardware
acceleration
Defined by a specification Rather than a de-facto
implementation
Rendering state machine Procedural Not a window system, not
a scene graph
No initial sub-setting Extensible
Data type rich Cross-platform
Window system-independent core
X Window System, Microsoft Windows, OS/2, OS X, etc.
Multi-language bindings C, FORTRAN, etc. Not merely an API, rather
a system
Later had OpenGL ES subset for embedded devices
CS 354 37
Evolved OpenGL Data Flow
vertexshading
rasterization& fragment
shading
textureraster
operationsframebuffer
pixelunpack
pixelpack
vertexpuller
clientmemory
pixeltransfer
glReadPixels / glCopyPixels / glCopyTex{Sub}Image
glDrawPixelsglBitmapglCopyPixels
glTex{Sub}ImageglCopyTex{Sub}Image
glDrawElementsglDrawArrays
selection / feedback / transform feedback
glVertex*glColor*glTexCoord*etc.
blendingdepth testingstencil testingaccumulation
storage
operations
CS 354 38
Buffer-centric Evolution Data moves onto GPU, away from CPU
Apps on CPUs just too slow at moving data otherwise
Vertex Array Buffer Object (VaBO)
Transform Feedback Buffer (XBO)
Parameter Buffer Object
(PaBO)
Pixel Unpack Buffer (PuBO)
Pixel Pack Buffer (PpBO)
Uniform BufferObject (UBO)
Texture Buffer Object (TexBO)
Vertex Puller
Vertex Shading
Geometry Shading
FragmentShading
Texturing
Array Element Buffer Object (VeBO)
Pixel Pipeline
vertex data
texel data
pixel data
parameter data
glBegin, glDrawElements, etc.
glDrawPixels, glTexImage2D, etc.
glReadPixels,etc.
Framebuffer
CS 354 39
Extensibility
OpenGL is evolved via extensions Legalistic—expressed as “amending” OpenGL’s
detailed specification Coordinated through Khronos
But vendors don’t need any blessing or permission to make extensions
Allows priorities to be shaped quickly Example: Direct3Dism extensions have greatly
reduced functional incompatibilities keeping Direct3D apps migrate to OpenGL
All the latest GPU features available in OpenGL Example: Tessellation shaders
CS 354 40
framebuffer
updates
Software and Hardware Architectures Incorporating 3D Pipelines
X Window System GeForce 6 Architecture
X server
Graphics Kernel driver
GPU Hardware
OpenGLdriver
Application
OpenGL library
GLUT
X clientlibraries
Operating systemkernel (Linux)
vertexprocessing
rasterization
shading
CS 354 41
Hardware Platforms depending on the OpenGL Architecture
Workstation PCs
Consumer PCs
High-end Visual Computing Solution (VCS)
Embedded Applications
Handheld Devices
Game Consoles
ConventionalPC
OpenGLProducts
Unconventionalnon-PCOpenGL
platforms
CS 354 42
Student’s View of OpenGL
You can learn OpenGL gradually Lots of its can be ignored for now The “classic” API is particularly nice
“Deprecation” has ruined the pedagogical niceness of OpenGL; ignore deprecation
Plenty of documentation and sample code Makes concrete the abstract graphics
pipeline for rasterization
CS 354 43
Next Lecture
3D Viewing How are 3D scenes processed so they can be turned
into an image on the screen? Expect a short quiz on today’s lecture
Easy questions
Assignments Reading
Chapter 1, pages 1-38 Makes sure your CS Unix account is active
Homework next time will be compiling a simple OpenGL example