Meet the team behind the NFL's on-screen first down line

Durham-based SMT Technologies is known for making possible the visual aids you see in sports TV, like the first down line in NFL games.

DURHAM — Stroll through an art museum or an art gallery where painters and sculptors display their works. Some artists dream of having tens of thousands of people view their creations. 

Then there’s Ian Prokes… 

More than 18 million people see Prokes’s creations every week. 

“It takes a lot of focus and I have a good team with me to keep me straight,” Prokes admits. “I’m a little exhausted and stressed by the end of the game. The good thing is I limit the coffee intake before a game starts, that's for sure.” 

Prokes is a remote operations technician, blending technology, science and art to draw the first down line on Sunday Night Football, the NFL’s biggest game of the week.  

That’s right, the iconic yellow line. It’s all up to him and his team. 

“So you have to make sure you have your ball positioned and everything is ready to go before they come back to the game camera otherwise the line will come in late,” explains Prokes, who adds that his job is especially difficult at certain times during a game. “Especially if it is in a two minute warning or a team is running a hurry up offense,” says Prokes. “That’s when you have to do a lot of things before the next play.” 

The first down line may be the best-known work of Durham-based SMT Technologies. But these days the company adds visual aids for most televised sports, including strike zones in baseball, pointers for NASCAR drivers and the flight paths for golf balls. 

“For us it really is about informing the viewer,” says John Dengler, a vice president of software technology at SMT. Dengler was one of the company’s first designers. “It has to start with providing information, something that gives the viewer a piece of information they wouldn’t have otherwise and putting it in a place where they can see it easily.” 

SMT is essentially leading a television revolution. The company developed the software to take live data and immediately put it on the air. Think of the leader board in a NASCAR race that is constantly updated. But while technology now allows real time data to be displayed, viewers must be comfortable with graphics constantly on the screen. Once it became clear that viewers enjoyed seeing a constant stream of information on the screen, the next step was to have graphics virtually inserted into a scene. 

So just how does the first down line happen? The TV magic is in the mathematics. 

There are 19 different elements in the lens of a television camera used to broadcast an NFL game. SMT technicians designed software to get readings from the lens 60 times/second, telling computers how the lens is zoomed and where it is focused. 

Technicians also get a reading from the pan head of the camera. That’s what connects the camera to its base. Those readings tell computers how the camera is tilted and how it is panned.

“Those readings tell us exactly what the camera is looking at,” explains Hans Weber, vice-president of research and development at SMT as he points to a television screen. “So in this case, the camera is looking at an area from the right 45 yard line to the left 40 yard line and it is zoomed in to see basically the width of the field.” 

That information is used to build a mathematical model of how the camera sees the field. 

The information is then combined with another model of where images need to be put on the field. 

“I want to put a line of scrimmage here at the right 46 yard line and I want to put a first down marker at the right 36 yards line,” says Weber, indicating the lines drawn on the screen. “And together with those two models we can figure out where to draw those elements. 

The data is relayed from the camera lens and the pan head 60 times per second, which is the same rate for the images of where the lines should be drawn on the field. Because so much data is reported so frequently, the first down line moves with the camera. 

But there’s something else that goes into making the first down line appear as if it is on the field, behind the players and a part of the game. And that’s color. 

“In addition to the information from the camera and the scene, what makes this all work is how we analyze color, for example the green on the field,” says Weber, gesturing over the field displayed on a screen. “But you can see there are different greens, so the operator needs to pick different areas and identify a set of colors you are willing to draw on top of, for example, all of the shades of green. But you also have to identify colors you are not willing to draw on, such as the white of the uniform, the black on the helmet or the yellow of the pants.” 

The algorithm is sophisticated. It identifies various hues, saturations and shades of a color; in this case, the green of the field. 

It takes technicians about three hours to set up the cameras with the technology. There are usually six cameras used in an NFL game. 

But for all the technology, it’s the technician that must enter the information into the system in about ten seconds. You could call it high-pressure art!

Related Resources: