5 key requirements of successful big data projects
In her keynote presentation at the Apache: Big Data North America conference in Vancouver on Monday, Gaskins stressed that five factors can make or break big data projects:
"When we talk about requirements for succeeding, we think about Maslow's Hierarchy of Needs," Gaskins says. "But the truth is it's really a system and any part of the system can break down."
Gaskins, who most recently served as big data project director at the National Oceanic and Atmospheric Administration (NOAA), drew on three personal experiences to illustrate the point: helping the 43rd Sustainment Brigade in Afghanistan root out corruption that led to resources falling into the hands of the Taliban, helping MetLife's Dubai office build an automated solution for detecting insurance fraud, and helping NOAA open up and commercialize its weather data.
The first two projects met each of the five requirements she highlighted and proved successful.
In Afghanistan, Gaskins, once a military intelligence officer herself, served as an embedded mentor with U.S. Army Intelligence and Security Command (INSCOM). She was embedded with the 43rd Sustainment Brigade when its intelligence officer returned to the U.S. The brigade served about 5,000 soldiers, but the intelligence unit only had six people. Gaskins helped pioneer a program that used truck drivers and others to help gather intelligence that the team could analyze for evidence of corruption and bribery.
Working with MetLife in Dubai, Gaskins drew on insurance claims adjusters as SMEs to help build an automated solution to detect fraud that ultimately resulted in a 400+ percent ROI.
The third project had some success, but missed the mark when it came to buy-in from NOAA's political leadership. It also lacked a sense of urgency as a result. The project did successfully open much of NOAA's data to the public, though the organizations that have had the most success using it to date are ones that have poached NOAA's SMEs to understand the data available.
"It was an egalitarian style team with no titles, which allowed everyone to make decisions very easily," Gaskins says. "We were open, transparent and this made the team really safe."