Enterprise Data Warehouse Centralizes Operational Info for Key PANYNJ Airports

In a flash, the information that airports routinely collect about arrivals, departures, passenger counts, etc. has become more important than ever. Records detailing where each passenger originated, for instance, are crucial in light of COVID-19 travel restrictions and advisories.

But accessing or analyzing such information can be a tedious process. Too often, airports collect data from a variety of sources, and then it sits in separate silos with restricted access. When personnel need information, they must ask a specialist to retrieve and analyze it. Sometimes the process can take weeks or even months—costly and unacceptable delays when passenger health is at stake.

The Port Authority of New York and New Jersey (PANYNJ) recently found itself in an enviable position to navigate the pandemic’s unprecedented information technology challenges. About one year before the novel coronavirus emerged in the United States, the Port Authority had added a new cloud-based enterprise data warehouse and business intelligence system, known in the tech industry as Aviation Data Analytics Warehouse (ADAW) and Business Intelligence (BI), respectively. The centralized system coordinates information collected from multiple data streams and departments and allows authorized users to easily access the data. 

As a result, John F. Kennedy International (JFK), Newark Liberty International (EWR) and LaGuardia Airport (LGA) have all experienced dramatic benefits.

“Instead of the operations team working off one set of data, the maintenance team off another, and the customer service team working off yet another, everyone sees a single version of the truth,” explains Ai Yamanaka, data analytics program manager for the Port Authority’s Aviation Strategy Unit.

The improved data management system has been invaluable during the pandemic, adds Aviation Director Huntley Lawrence. “These technology advancements ensure a more effective way to collect, analyze and forecast data, which has proved critical for our response to the COVID-19 pandemic,” Lawrence explains. “The Aviation Department relies on our ability to gather and analyze data to coordinate daily operations with the airport community and develop predictive models for future passenger activity to provide the best experience possible.”

Rohun Iyer, senior data scientist from the Aviation Strategy Unit, created a custom dashboard that estimates and predicts how many passengers will arrive from states with travel advisories. Iyer’s new tool combines information from multiple data sources and information within the Port Authority’s data warehouse with updates from the COVID Tracking Project, an effort launched by The Atlantic that collects and publishes data about the deadly pandemic.

“The dashboard helped our operations team understand changes in passenger demand because of spiking COVID cases by state,” Yamanaka reports. “Our leadership uses the dashboard to inform data-driven decisions, such as how and when to deploy New York State Department of Health officials at our airports to survey incoming passengers in accordance with travel advisories.”

Skilled data warehouse users such as Brian Levine, PANYNJ’s manager of strategic analysis and forecasting, cross-analyze TSA data, flight schedules, operations information from individual airports and other key data to yield new insight. During the pandemic’s early months, Levine compiled weekly presentations that informed Port Authority leadership about expected changes in flight schedules and travel trends that accurately forecasted traffic and revenue, notes Yamanaka.

Iyer continues to refine the COVID dashboards and scripts to incorporate real-time hourly passenger data. The changes will allow airports to predict passenger flows and better manage COVID-19 protocols such as social distancing and enhanced cleaning procedures.

This success story is just a snippet of what PANYNJ can achieve with its new data management system, notes Yamanaka “Our ultimate goal is to improve the entire customer experience,” she says.

Program Development

The genesis of PANYNJ’s enterprise data warehouse and business intelligence system dates back to 2018, when Lawrence began his tenure as director of Aviation for the Port Authority. One of his original goals was to build a robust data analytics team to advance decision-making across the Port Authority and its airports.

His vision inspired PANYNJ personnel to solicit advice from leading data teams at the Metropolitan Washington Airports Authority, San Francisco International Airport and Los Angeles World Airports. The insight they shared helped PANYNJ craft a blueprint for a centralized data warehouse and platform—the first foundational step toward a more comprehensive data infrastructure system.

“We got a sense of their best practices and how each team started, then developed our own strategy,” Yamanaka explains. “Our system was a little different in that we wanted to create a centralized team to coordinate information across different airports and different airport teams. We have three commercial airports, and each has its own culture and operational channel. That also applies to how we gather information and the different information channels we use.”

To help pull off a project of such magnitude, PANYNJ partnered with REVISION Inc., the consultancy that built Denver International Airport’s Enterprise Data Management Platform and Adoption Program.

PANYNJ launched its new data management and analytics program in 2019.

Evolution vs. Revolution

Seeking to produce quick results on its initial investment, the Port Authority focused on adding as many production-ready data streams into the system as the budget allowed rather than immediately building an integrated and governed data warehouse. REVISION first built the back-end infrastructure of the data program as a series of data marts joined by a connecting key, database to database.

The work started with airside- and landside-focused datasets. The airside initiatives included information about flight delays, cancellations, on-time flight performance and other operational measures. The landside efforts included real-time customer survey data. The major initiatives driving the need were poor on-time performance, predictive flight activity to better manage LGA construction schedules and improving customer survey scores.

In the program’s first year, the data team integrated more than 10 new data streams into individual data marts. “We prioritized based on need and usability across departments,” says Iyer. “Some datasets were a bit more challenging to bring into the warehouse and others made sense to bring in.”

Yamanaka explains that before adding datasets, the team needed to understand where the data originated and how to migrate it to the new database. Some datasets were hidden behind firewalls and access restrictions, while others resided on airport desktops or in the cloud.

Analytics teams from REVISION and PANYNJ cleaned and vetted data before importing it into the new warehouse system to engender trust from subsequent users and avoid squandering time analyzing invalid data.

Yamanaka reports that initial efforts immediately showed a return on investment by helping teams produce dashboards from a common platform. Now, authorized airport users can pull any data stream housed within the warehouse into an automatically refreshing customized dashboard, report or Excel file.

Proving value early in the project allowed PANYNJ to build a back-end governed data warehouse—adding even more data streams—in the second year. All data streams are designed to share and map to a few key common tables within one database to allow for seamless integration across data streams.

Yamanaka uses the analogy of a large mansion to describe how it works. “Instead of having to open various doors to get in and out of various rooms, a governed data warehouse allows for one large loft with partitions, so users can move from one portion of the room to the next,” she explains. “This back-end governance work is in line with Airports Council International’s Aviation Community Recommended Information Services (ACRIS) best practices and is a key improvement that will provide our users with connected, governed and clean data.”

Though the rollout does not include all data streams, personnel can access the warehouse regardless of whether their team contributed data. “We taught them how to manage their data, create dashboards and generate insights from the other data streams,” Iyer says. “Even if their data isn’t in the warehouse, they can structure their information and connect it through a local connection to our front-end tool, giving them a way to leverage data analytics.”

Empowering Users

The analytics department used group classes, targeted education and one-on-one sessions to train various users.

Training on the front-end dashboarding platform included group lessons offered in-person and virtually by department. Employees can also view training session videos, presentations and “quick guides” on the team website.

Advanced training followed to foster even more independence. “Our data team cannot do everything itself. We cannot do all the reporting for every request,” Iyer explains. “The goal of our training is to empower folks to feel comfortable accessing the information, interpreting it, and only coming to us if they spot any discrepancies.”

After holding sequential training sessions for a full year, the team worked with “power users” from each unit who applied the skills they learned to their unit’s data streams and routine work. “We are focused on getting folks to understand the information and use the data in day-to-day decision making and operations,” Iyer remarks. “It’s a culture shift. We encourage the teams to use information for their decision making and have their staff own that information.”

That’s the main goal for any data management project, adds Zane Shults, senior vice president of BI and Analytics and Program executive with REVISION. “The program was designed to provide PANYNJ an industry leading self-service Enterprise Data Platform and the educational foundation to enable self-service analytics. The objective of this program is to get good, accurate and governed data into users’ hands and give them the ability to use it.”

Already, the strategy seems to be working. “A lot of dashboards are now being created by the users themselves, and they have become self-sufficient in their use of analytics,” Shults reports.

This culture shift frees up the data analytics team to work on more advanced analytics, such as developing predictive models from machine learning, and exploring new tools to address other challenges.

Leveraging Data

PANYNJ is measuring staff engagement, use and adoption to assess the program’s effectiveness.  Now in its third year of use, the new data management system is proving to be a valuable asset, reports Yamanaka.

For example, the customer service team uses dashboards to analyze data from Airport Service Quality surveys and then presents its findings to terminal operators at JFK, EWR and LGA. “By analyzing the data, the customer experience team understands the pain points that matter most to our customers and can track trends of improved customer experience in those areas,” Yamanaka explains.

Airport Operations Center (AOC) teams use customized dashboards to coordinate daily traffic. LGA, for example, leverages flight schedule data to develop passenger flow estimates and better manage demand at its terminals and frontages. “Knowing how many passengers are coming in helps them to prepare,” says Phil McDonough, senior vice president of Transportation at REVISION. “It allows them to be proactive instead of reactive in their operations, both airside and terminal.”

Such data is especially valuable when airport officials must align operations with ongoing construction efforts, he adds. For instance, the data team helped LGA’s operations team develop a dashboard that tracks delayed flights and how their arrivals align with construction schedules. Operations personnel meet every afternoon to review traffic, see when flights will arrive, and determine whether they need to divert them. “They used our dashboards to work with terminal operators and airlines to mitigate customer experience impacts by deciding which flights to accommodate and how best to optimize the construction schedule,” Yamanaka says.

An Eye Toward the Future

In general, the goal of most data programs is to introduce business insights that can help teams realize operational efficiencies and increase customer service. As PANYNJ’s data program matures, Yamanaka sees teams shifting their focus from reporting to analytics. 

“As our business users become more comfortable using data in their day-to-day operations and work, the use-case questions have evolved from reporting on past performance to leveraging historical and real-time data to make decisions and operate more strategically,” she explains. “For example, the Air Service Development manager now uses [enterprise data warehouse] information when meeting with potential entrants into the airports. This gives a potential carrier insight into untapped markets and route demand based on our origin-destination pairs and bookings, using data to spur competitive air service development at our airports.”

New uses continue to crop up every day, adds Iyer. Currently, he is working with the financial services team and comptroller’s office to develop a dashboard that will help project upcoming takeoff and landing fee charges. The goal is to streamline monthly revenue projections across teams and to help anticipate the need to change airline rates and charges.

Overall, PANYNJ has found that having a central place to collect, analyze and forecast data is critical for timely response to operational issues. “One of the hardest things about data analytics is making sure people are telling the same story with the data,” says McDonough. “The Port Authority’s new system ensures that no matter how they slice and dice the data, the story stays the same."