Mineta San Jose Int’l Partners with U.S. Customs & Border Protection on Facial Recognition Systems to Screen Arriving & Departing Int’l Passengers

As the gateway to Silicon Valley, it’s natural for Mineta San Jose International Airport (SJC) to leverage technology, researched and developed by local tech companies, to solve everyday passenger processing challenges. Of late, the California airport is using biometric facial recognition technology to screen international passengers and is leading the way for other U.S. airports to follow. 

Explosive traffic growth that tripled the number of international flights since 2013 had led to unacceptable passenger processing delays. With four gates for international arrivals, passengers from multiple flights sometimes flooded the U.S. Customs and Border Protection (CBP) facility simultaneously.

“There were times when we had three to four aircraft at those gates; some wide-body and some narrow-body,” explains Bob Lockhart, SJC’s deputy director of operations. “We would have 700+ people trying to go through the facility at the same time. People ended up waiting for an hour or two. We had one occurrence when people waited over three hours.”

The backups put the airport at risk of losing international passenger traffic to two nearby competitors: Oakland International Airport and San Francisco International Airport. As such, it became essential for SJC to change its approach. “In our highly competitive market, it is crucial that Silicon Valley’s airport continue to be the region’s airport of choice, given the convenience and on-time performance we are known for,” says Director of Aviation John Aitken. SJC served 14.3 million passengers in 2018, a 63% increase over five years.

To address its processing delays, the airport sought a partnership with CBP—just when the agency was developing parameters for entry and exit programs using biometric facial recognition technology. The result is a new passenger processing system in the airport’s CBP arrivals facility and at SJC’s international boarding gates. The entry project was funded by Congress, which authorized CBP to spend up to $100 million annually for biometric installations at airports. SJC committed to supporting entry and exit programs, and funded the exit biometrics program for a multiple airline, common use, one-step process. As such, SJC was one of the first airports in the country to commit to processing all international travelers with facial biometrics. 

The new systems, which went live in two phases, expedite the processing of international travelers. Phase 1 was launched in June 2018, with CBP implementing biometric facial recognition screening for arriving international travelers. The process for departing international travelers debuted this February with SJC partnering with ANA as the pilot airline. 

“Our airport wants to be on the cutting edge of technology, so it was important that we be an early adopter and investor to support our tech-oriented travelers,” Lockhart says. “I think we are the right-size airport with the right amount of flights to give the CBP a good testing bed to prove this technology can work, just as easily as at the larger airports. And, it’s something passengers see as an easy, simple and efficient process.” 

The new system has produced impressive results, he reports. At a press conference after implementing biometric entry for a six-week period, the CBP demonstrated that each passenger is now typically processed in less than two seconds and with a 99% match rate, with the added benefit of eliminating the need to scan the traveler’s passport. 

Despite being very new, Lockhart predicts Phase 2 will provide similar improvements for departing passengers. “The first phase has been very, very successful in processing people quickly and reducing wait times, and I expect the same results with exit,” he says. “The biometric process is so quick at reading faces and getting information back.”

CBP officials are also optimistic about the final phase. “We expect it will be more efficient than the current process and more secure because of its imposter detection capabilities,” says Dan Tanciar, deputy executive director of the agency’s Planning, Program Analysis and Evaluation Office. 

The airport anticipates that making the process more efficient may save money. Cost estimates associated with the 559 Reimbursable Services Program that SJC uses to pay overtime to CBP officers run from $1 million to $1.5 million per year. While biometrics may not totally eliminate these expenses, they will help in managing the need for CBP overtime. 

The Road to Facial Recognition

CBP’s first foray into biometrics began in 2004, when it began fingerprinting foreign nationals as they arrived to the U.S. “But we just couldn’t get our heads around how to implement fingerprinting with outbound traffic in a way that didn’t grind travel to a halt,” recalls Tanciar. 

While European airports operate separate international departure terminals, that’s generally not the case at U.S. airports. A flight destined for Kansas might depart next to a gate with a flight bound for Beijing, China. “Our airport infrastructure just wasn’t built for a final biometric mission involving fingerprints,” he explains.

But in 2016, CBP found that if it leveraged biometric information and photos on travelers’ passports, it could use facial recognition technology to expedite the process. “We thought, ‘What if we took a quick picture of people as they passed by and matched each person to a passport photograph in the Department of Homeland Security (DHS) database?’ We loaded the images of passengers for a single flight onto a tablet, hooked it up to a camera and tested the idea, and lo’ and behold, it worked,” he explains. 

This revelation led CBP to develop a cloud-based solution that facilitates processing all international arrivals and departures with facial recognition technology. The CBP then decided to work with select airline and airport partners to transition the technology into airports, rather than mandate the process for all. 

Tanciar explains that airports have unique limitations and size constraints, and airlines like to operate in a way that’s conducive to each. “Rather than go forward with a government-mandated solution for facial recognition, we gave some parameters by which the process and equipment should work, then let the airports identify what works best for them. San Jose was one of our first supporters in this endeavor,” he comments.

The goals for the partnership were simple, adds Tanciar: 

1. Congress mandated inbound and outbound biometrics, and CBP sought to meet that requirement. 
2. The system needed to be fast, effective and efficient. “We had to ‘do no harm’ in terms of how aircraft are boarded. We knew we wouldn’t get much more than two seconds per traveler to accomplish this biometric mission,” he recalls. “We had to be able to implement it in a way that didn’t delay departures.” Adding fingerprints in 2004 represented a significant security achievement but added about 90 seconds to every inbound transaction for foreign nationals.
3. The resulting system had to be accurate in matching the right people to the right document. 

Rather than starting the process by scanning passports and collecting fingerprints for every international traveler, the new biometric system takes photos of arriving and departing international passengers as they walk by. The system then checks the photos against images on file from the DHS database of passport and visa photos. If passengers’ basic information and airport photos match what’s in the database, they continue on their journeys. 

“It is a huge time savings from an inspections point of view because we’ve eliminated those administrative steps of handling and scanning the travel document,” Tanciar says. “This all happens in less than two seconds, so that the minute-thirty that was added with fingerprints has now been subtracted again. I predict you’ll see arriving processing times for foreign nationals that are pretty close to those of U.S. citizens.” 

Using biometrics meters departing passengers better on the boarding bridge, adds Lockhart. “This helps get people on the plane faster,” he reports. “They are boarding naturally, and everyone has time to do what they need to do before they sit down.” 

The system’s speed and 99% match rate are made possible because it does not compare each passenger’s image against the entire DHS database. Instead, CBP receives smaller galleries of passport and visa images based on each flight manifest.

If for some reason a passenger’s image does not match the one housed in the database, the passenger goes through the traditional processing procedures already in place. An airline gate agent asks the traveler for a boarding pass, scans it and reviews the travel document. If everything matches, the traveler boards the aircraft. If the gate agent suspects the person is using someone else’s passport, CBP takes it from there.

“We’ve had some great successes in identifying imposters,” Tanciar reports. “At Washington-Dulles International Airport, the system identified three imposters in 40 days. It really helps us achieve what we need to do—find folks who should not enter the United States or wish to do us harm.” 

Tackling the Tech 

SJC partnered with Amadeus Airport IT Group for its new facial recognition system. Lockhart explains that the choice made sense, because the airport uses the company’s common-use systems throughout its two terminals—flight information displays, boarding pass gate readers and various equipment at airline counters. By expanding the company’s boarding pass readers with Gemalto facial recognition technology, the airport’s goal is to tie biometrics into the boarding process, streamlining passenger processing to a single step. 

After the system captures a passenger’s face, it compares the image against a database of images for the flight. Gate agents watch a screen for either a green checkmark that confirms the passenger’s identity or a red stop sign that indicates the individual needs to talk to an agent. 

Betros Wakim, president of Amadeus Airport IT Americas, explains that the system required two points of integration. One involved taking the image and sending it to CBP and getting results back. To do this, developers incorporated two Gemalto facial recognition cameras that triangulate and capture an image as each passenger walks by. The system then leverages the company’s image processing software to securely transmit the data to CBP. “Passengers don’t have to stand there and pose for a picture,” he says. “They don’t have to hand their passport or even their phone [loaded with a boarding pass] to the agent. The idea is to remove all frictions from the travel journey so passengers can have a frictionless trip.” (See sidebar on Page 46.)

The second part involved integrating systems to transport data back to the common-use system with the traveler’s identification number, which gives airline agents clearance to board that passenger.

To ease implementation, SJC and Amadeus rolled out the exit system with ANA first, because it also operates the company’s Altea system. Once the process is fine-tuned with ANA’s daily nonstop to Tokyo-Narita, SJC will expand the technology to all international airlines. “If the airline uses third-party software, where we don’t have a relationship, integration will be a little bit harder,” Wakim explains. “But because of our background, we’re able to integrate the systems and still deploy this capability as a single step.” 

Passenger Privacy

The entire system had to comply with the EU General Data Protection Regulation, which is widely considered the most significant change of data privacy protections in 20 years. Because the facial recognition technology uses an image and other personally identifiable information, processes were needed to protect the data and delete it from the cloud shortly after use. In contrast, CBP retains fingerprints and photographs of most foreign nationals for up to 75 years. 

“We knew early on that whatever we put into the cloud to enable fast, accurate matching and responses back to the airline for departure had to be minimal. We had to minimize the amount of personally identifiable information that’s available, so [gate agent and CBP officers] receive a templated photo, which is encrypted and associated with a unique identifier tied to the airline manifest—that is all that is stored in the exterior cloud. There are no names, birthdates or other personal information being transmitted,” Tanciar explains. “We looked at the privacy aspect of this very mindfully, and security-wise, we have a great cybersecurity division at CBP, which helped us build a resilient system that protects against cyber threats.”

Airline agents do not see the whole image of a passenger, adds Wakim. Instead, they see coordinates and numbers. The system isn’t taking the whole dimension, it’s taking characteristics of the face and sending it to the cloud, he explains. 

What’s on the Horizon? 

Currently, SJC has installed facial biometric systems at two international gates, with deployment to soon follow at two more. Mobile systems will also be installed at up to three additional gates. “It’s important that we have global capability and flexibility on as many gates as possible,” says Lockhart. 

On a holistic level, he sees the future as very bright for this technology and anticipates SJC may eventually use facial recognition technology for every flight, even domestic departures. More near-term, CBP and the airport hope to add cameras that can photograph groups of international travelers, such as families with small children, at the same time. 

“Currently upon arrival, they have to walk up separately to get their photos taken by Customs agents, but CBP will eventually roll out a system that displays multiple pictures on the screen at the same time. SJC would like to offer the same for departing international passengers,” says Lockhart. “That way, CBP and airline agents can look at them and approve them all simultaneously, which will speed up processing dramatically because processing a family of passengers will take 30 seconds total instead of 30 seconds per person. 

“Right now, we are on the leading edge with facial recognition technology, but eventually it will become the norm,” he adds. “There will be no more retrieving your passport and boarding pass multiple times, while juggling luggage and children.”