Transporting analytics to the Internet of Things

By IDB Guest Blogger: Lee Ann Dietz, SAS

Why are so many companies across a diverse set of industries investing in and around the Internet of Things? Everywhere I go, every blog I read … I sound like my favorite band from the 80s: the Internet of Things is watching me.

In reality, it’s the reverse: I’m seeing the Internet of Things (Iot) everywhere: companies investing in sensors, networking and applications with the expectation that this investment will increase revenues, lower costs and improve profitability over the short and long term.

While the term the Internet of Things was coined in 1999 by Kevin Ashton at Procter & Gamble, the mainstream application of IoT is just getting started. As the trend has heightened, I’ve been evaluating the potential for IoT to support better decision making in travel and transportation.

My experience in the travel and transportation industry has always been about using analytics to support decision making. In fact, I started my career as a pricing strategy analyst at American Airlines. And now I’m fully converted. You can call me an evangelist for the IoT in transportation, especially because the potential to take data coming from the IoT, incorporate purposeful analytics and reach better decisions quickly, is significant.

This week, I am presenting as a guest of the Institute for Defense & Business at the 2015 NDTA-USTRANSCOM Fall Meeting on the convergence of IoT and analytics in the transportation industry.

I am speaking directly about why I have become so enamored with the ability of the IoT to deliver business value, especially in transportation.  It’s really quite simple: the IoT delivers value through the data that the things provide for decision-making.

This data is collected by sensors and devices on railcar components, semi-truck engines, or other elements within the transportation value chain. And it is now available on a real-time, or streaming, basis.  This provides the ability to learn from trends in that data quickly and act upon those trends within seconds.

Gone are the days where data is collected over the span of weeks or months, sent to an analyst for review, and – after another week or two of data crunching – the analyst presents a report or PowerPoint to her boss with recommendations for changes.

As Tom Davenport said:

To make the Internet of Things useful, we need an Analytics of Things.  This will mean new data management and integration approaches, and new ways to analyze streaming data continuously.

So, in order to take advantage of the streaming big data (and it is big data by every definition of the phrase) coming from the sensors, we must reconsider how we use analytics.  Remember, that I didn’t say that the analytics themselves must change.  In most cases, we can use the same analytics applied to streaming data as we used in a batch model.

What we need is a good understanding of where we apply the analytics: on the edge, at rest or in the middle. Let me explain:

  • Analytics on the edge means analysis at the specific device or sensor.
  • Analytics at rest means data pulled out of the stream and used for high-performance analytic model development
  • Analytics in the middle takes place on data as it’s streaming. Some analysts have called this middle ground “the fog,” and it’s relevant because it can be a combination of the streaming data itself enriched with sitting data such that we can detect more complex events sooner.

We have now arrived at a different place in the analytic continuum.  The optimal analytics experience is a multi-stage analytics experience.  It includes continuous queries on data in motion and at the edge, with incrementally updated results.  This new process moves analytics from centralized data warehouses to edge analytics, which are closer to the occurrence of the events.

What does multi-stage analytics of IoT data look like? It happens fast (seriously, we are speaking about microseconds or msecs at this point) and at very high volumes.  It requires specific business rules that give instructions on whether to save, discard, aggregate, transform or enrich the streaming data without overloading the entire system or network.  Multi-stage analytics includes pre-determined data mining, decision making, alerting, scoring, and profiling of the data to exploit the value of the streaming data.  And, it might also include managing the data differently – creating “out of order” handling to make the data source streams understandable to the analytics and the decision-makers.

We have all the building blocks in place to exploit the value of the Internet of Things and the Analytics of Things: sensors, or assets, creating data, the communications network connecting the data and the analytic and computing applications that make use of the data flowing to and from the things.

The Internet of Things can be transformative in transportation operations: in maintenance and engineering, you will have more information sooner, which means you can predict the maintenance needs of individual assets before failures occur and proactively service assets at an opportune time when your asset is near a repair facility.  This reduces costs across your operations.  In supply chain situations, you can monitor inventory levels on a near real-time basis, develop better forecasting models and optimize this inventory, when and where you need, lowering supply costs, increasing efficiency and enhancing revenue opportunities. On the customer side of transportation, you can enhance the customer’s experience by providing real-time forecasts of arrivals and notifying them sooner if delays occur.  And, happier customers are loyal customers.

The Internet of Things opens up tremendous opportunities for transportation companies, generating significant streaming data which can be relevant for decision-making.  However, it is critical to apply the appropriate analytics to streaming data in order to derive value from that data.

Multi-stage analytics is not rocket science; it’s simply the judicious application of the right analytics at the right time in the right place to the right data, which is what you need to exploit the value of the Internet of Things.  That is why I’ve become an Internet of Things and Analytics of Things evangelist.


This content is reposted with permission from SAS Voices, where the original post appeared.

Students Get a Taste of Government Analytics at IDB Hosted Panel

By IDB Guest Blogger: Gail Bamford, SAS

When Tom Davenport and D.J. Patil suggested in their article published in Harvard Business Review that the data scientist is the “sexiest profession of the 21st century,” these in-demand professionals became part of the discussion surrounding the big data ground swell.

On April 10, 2015 at the 3rd Annual Business Analytics Forum at Indiana University’s Kelley School of Business the Institute of Defense and Business (IDB) and SAS had an opportunity to address over two hundred students, faculty members and industry partners who have a vested interest in grooming these superstars of the future.

The Kelley School’s Institute for Business Analytics , co-chaired by Vijay Khatri and Frank Acito, was one of the first programs of its kind established to prepare students for careers in business analytics. The Institute hosts this conference to make their students savvy in real world analytics and bridge the gap between academia and the real world. They actively seek out and engage with industry partners who provide internships and other pathways to employment.

This year’s conference topics included the Internet of Things (IOT), supply chain analytics, and healthcare analytics. The IDB, SAS, Deloitte and IBM were invited to inject the importance of analytics to government.

sas infographic government analyticsGovernment Analytics Jobs

Government is a heavy user of analytics and acutely feels the pain of the analytics skills gap across all agencies. A 2014 GovLoop survey reported that 96 percent of those surveyed – 46 percent of whom self-identify as experts or analysts – believe their agency has a data skills gap

Last year’s audience gave the thumbs up to our “Data vs. Gut” panel discussion, which featured retired military officers giving their perspective on the evolution they’ve seen of analytics use in decision making in defense organizations. This year, due to unforeseen circumstances, our defense panelist was not able to make it. Van Noah, panel moderator, smoothly shifted gears and reworked the flow of the conversation to provide the students in the audience with a broad preview of how they can apply their data skills in government analytics jobs.

If you were not able to see the discussion in person, here is a summary of the content these experts were able to shed light on:

  • Since this panel discussion was held just five days before the income tax filing deadline of April 15th – and stories about scammers were prolific in the media — what better subject to open the conversation?
  • Indiana University Kelley School of Business 04.10.2015Van Noah from IDB was able to represent the government sector and weave in some thoughts on how the military could better use analytics to select candidates for intensive training programs. For example, pilot training is expensive, so using analytics to identify students who are likely to wash out early is much safer and more cost effective than letting all students progress to the next phase of training.
  • Satish Lalchand from Deloitte has a long history of working with analytics in government and provided examples of how the federal government is leveraging analytics to combat fraud and improper payments. He also shared his perspective on how agencies can get started once they see the business need for analytics and what it takes besides analytics to make better decisions.
  • Eric Zidenberg from SAS has been involved with public safety organizations for many years and talked about how analytics are currently being used on the southwest border to make better decisions on which cars should be sent secondary inspection for illicit materials.
  • Dion Rudnicki from IBM segued nicely into talking about how the Memphis Police Department was using analytics to better allocate resources to reduce crime. In a city previously identified as #1 for crime in the US, and where there are only 2,500 police officers to protect the population of 650,000, analytics allows government to place the right resources at the right places at the right time. The result: a 30% drop in crime.

Next Generation Data Scientists

Students preparing for careers in data science have a lot of options when they enter the workforce. The data analytics talent gap exists in every sector in our global economy. When students weigh their options, they should consider the jobs that support the public sector. Serving government will give them a chance to make a real difference. Government needs these talented, data scientists of the future!

Gail Bamford is an IDB guest blogger and has over 30 years of experience working in the public sector IT market. She has been with SAS since 2006 and is passionate about helping close the analytics talent gap. For more than a decade, SAS and the IDB have worked collaboratively to raise the level of awareness of the value analytics to defense leaders.

Data, Gut, Watson, and Angelina Jolie

The fourth and last of the Data vs. Gut panel series took place at the 2014 North Carolina Federal Advanced Technologies Review (NCFATR) conference on June 5. Scott Dorney, Executive Director of the North Carolina Military Business Center invited the Institute for Defense and Business (IDB) and SAS to give a repeat performance of the original Data vs. Gut panel discussion last fall.

The discussion was moderated by IDB president, Mark Cramer, and featured Karen Terrell, VP of SAS Federal and former IDB Executive Fellow, Rear Admiral Erroll Brown (USCG ret.).June 5, 2014 NCFATR Panel "Data vs. Gut"


In every panel discussion we’ve hosted on Data vs. Gut, the importance of data leads the discussion. Good data is critical to successful analytics. It’s not surprising that this comes up over and over since there are many examples of bad data leading to disastrous decisions with disastrous consequences. In doing analysis, be mindful of:

  • Data quality: How accurate is your data? Accurate data analysis is only as good as the data put in. A lot of data is still entered manually and is incomplete or incorrect.
  • Timeliness: How important is timely data in your analysis? Time-critical decisions depend on easy access to the most up-to-date data.
  • Data relevancy: What data do you need to solve your problem? You don’t have to analyze every bit of data you have access to. What data is relevant to your situation?

Brown had some observations and suggestions.

  • Problem: Clearly understand the problem you are trying to solve.
  • Data: Is the data right and is it the right data? Make sure you know your data. Make sure it is accurate and timely. And, make sure the data you are using in your analysis is relevant to the problem you are trying to solve.
  • Tools: Get the right analytic tools for the job. One size does not fit all. You have to have the right algorithms, or “algos” too.

Evolution of Data and Analytics

June 5, 2014 NCFATR Panel "Data vs. Gut"Terrell shared with the audience her observation of how the usage and value of data has evolved. In her 20+ years of serving the government market, she has seen customers move from basic reporting and descriptive statistics to advanced and predictive analytics. Government routinely uses analytics to detect and prevent fraud these days. Other areas trending up are predictive asset maintenance and identification of high risk procurements.

She also noted the “democratization” of analytics. If you know your data and know your business, you can unlock insights in your data with powerful visualization tools that make it possible to slice and dice your data from your desktop. It’s easy and you don’t have to be a statistician to understand the graphic output.

High performance analytics is allowing more data to be analyzed faster and with a greater degree of granularity. Terrell mentioned a large department store chain that is using analytics to do markdown optimization on 270 million SKUs in 850 stores. Optimization models determine which items to mark down and by how much. What used to take 30 hours to run, now takes less than 2. This saves the company millions of dollars every year in unnecessary markdowns.

Data vs. Gut

Terrell brought the data vs. gut discussion to a very human level when she mentioned Angelina Jolie, who saw both her mother and aunt die at early ages from cancer. In her NY Times Op Ed piece she talks about the reality and her decision. But, her radical decision was not made in a vacuum. She did her research and gathered the data. She carries the “faulty” BRCA1 gene and found out she had an 87% risk of developing breast cancer. With this knowledge, she was able to make the difficult, but data-driven decision to have a preventative double mastectomy.

June 5, 2014 NCFATR Panel "Data vs. Gut"Brown cautioned about machine-to-machine decision making. “Computers don’t make you smarter.” That’s where gut comes in. Humans must have a symbiotic relationship with machines. But, Brown, who spent several years at IBM after his retirement from the Coast Guard, couldn’t resist bringing Watson into the conversation. Watson learns like an individual and can tell you how it got to an answer or outcome. That’s more than he says he can get out of his seventeen-year old when he has confronted him with “How’d you get to that (stupid) decision?”

So if Watson processes data more like a human than a computer and learns as it goes, what will the data vs. gut discussion be in 10 years?

Gail Bamford is an IDB guest blogger and senior marketing professional with over 25 years in information technology. She has been with SAS, the leader in advanced analytics software, since 2006 and supports business units focused on delivering analytic solutions to Defense, National Security, Higher Education and K-12.

Additional Resources:

Download white paper based on Data vs. Gut panel #1 – “How Analytics Improves Decision Making at the Department of Defense: Finding new ways to add value and insights to big data.”

Read Blog on Data vs. Gut panel #2 – “SAS – IDB Defense Panel Serves Up Advice for Analytics Students at Kelley School Business Analytics Summit 2014”

Read Blog on Data vs. Gut panel #3 – “Shining Light on Data in Austere Environments”

Big Data for National Security

A few weeks ago, during the Special Operations Forces Industry Conference, the Institute for Defense & Business in conjunction with SAS Institute sponsored a luncheon panel discussion about Analytics and the Future of SOF. They offered me the opportunity to partner with Ambassador (retired) David Litt and Major General (retired) Kevin Leonard, U.S. Army to discuss how analytics can be used as an enabler for SOF operations in austere areas like Africa.

We’ve all heard the hype about “Big Data.” But SO WHAT? It’s generally discussed in the context of marketing, with one of the favored examples being Target identifying which of their customers are pregnant through their buying habits so that they can then send them coupons for diapers. But if you are not trying to sell diapers, you have probably missed out on the conversation, since selling diapers is a far cry from national security.

So instead of discussing “big data,” why don’t we discuss “Predictive Battlespace Awareness” or “Intelligence Preparation of the Operational Space”? These are two topics that are highly relevant to National Security. Funnily enough, both of these discussions lead us right back to big data, analytics, and the subject of last week’s conference, how can analytics be an enabler for SOF in the future.

Predictive Battlespace Awareness is the concept that you can develop a situational awareness picture so rich it enables you to anticipate what will happen so that you can develop course of action to thwart your adversary before they can act. A recent blog on my website discusses agribusiness in Africa as a sample use case. Achieving situational awareness is one element of the decision cycle: Observe, Orient, Decide, Act, also known as the OODA-loop.

For years, in all industries, the defense community not withstanding, we’ve been making huge investments into sensor technology to enable the observation phase of our OODA loop. And now we find ourselves overwhelmed with that very same data. We are no longer looking for a needle in a haystack, we are looking for a specific needle in a needlestack, and, as one might expect, it’s pretty painful.

Analytics technologies allow us to explore that data in a way that empowers us to advance our decision cycle through the orientation and decision phases. This means we can act more quickly when opportunities present themselves, respond more rapidly to thwart our adversaries, and anticipate where problems might develop so we can put remedial measures in place ahead of time.

Further, big data analytics capabilities can act as an enabler to collaboration. In an age of complexity, the next operation may require cooperation across multiple nations, non-governmental organizations, and both civilian and military authorities. Big data can facilitate this, even despite the fact that each organization runs on their own siloed system and processes.

In each of these cases, Special Forces will be able to leverage huge amounts of data to develop an intelligence picture of their operating space, collaborate with a coalition of any size and shape, and build the network they need to manage the complexity that is a part of normal operations.

This essentially recaps the discussion between David, Kevin at the discussion panel. Once again, I’d like to thank SAS and the IDB for the opportunity to share ideas. Big data may have us swimming in the ocean, but analytics capabilities offer a huge return on investment, not just for people looking to sell diapers, but for those of you who work in the national security community—in fact, the return on investment from a national security perspective is infinite!

Lt Col Tammy Schwartz, USAF (Ret.)Lt Col Tammy Schwartz, USAF (Ret.) is an IDB guest blogger and recognized innovator with more than 20 years of national security experience. Schwartz is a former Chief Technology Officer for Air Force Enterprise Networking and is currently a Consultant for SAS and the Owner of Llamrai Enterprises. You can read more from Schwartz on her blog.

Shining a Light on Data in Austere Environments

SAS and the Institute for Defense and Business (IDB) co-hosted their third panel discussion May 21st during the Special Operations Forces Industry Conference 2014. Retired Ambassador David Litt, head of the Center for Stabilization and Economic Reconstruction (CSER) program at the IDB, moderated the lunchtime discussion entitled, “Supporting SOF Network in Austere Environments: Using Analytics to Prepare, Sustain and Support.” In this discussion, the “austere environment” meant Africa.

The panelists, MG Kevin Leonard, USA (ret.), and Lt Col Tamara Schwartz, USAF (ret.), had interesting perspectives given their different backgrounds, but they reached similar conclusions. While technology, data, and analytics are enablers, it is TRUST, PARTNERSHIP, and COLLABORATION that make it work. The challenge is busting old paradigms and thinking outside the box.

From left to right: AMB (Ret.) David Litt (CSER), MG (Ret.) Kevin Leonard (Fluor), and Lt Col (Ret.) Tamara Schwartz (SAS Consultant) discuss data in austere environments during the May 21st panel sponsored by IDB and SAS

From left to right: AMB (Ret.) David Litt (CSER), MG (Ret.) Kevin Leonard (Fluor), and Lt Col (Ret.) Tamara Schwartz (SAS Consultant) discuss data in austere environments during the May 21st panel sponsored by IDB and SAS. (Institute for Defense and Business Photo by Christine Reynolds)

The Panelists

Leonard knows AUSTERITY and LOGISTICS. During his long career with the Army, he’s been involved in logistics activities in Afghanistan, Jordan, and Iraq – just to mention a few countries. He has supported BIG ARMY and the little Army and is continuing to support contingency operations at Fluor. He knows the challenges of getting supplies over the last mile. He also knows that if Coca Cola can do it, it can be done.

Schwartz is well acquainted with IT and enterprise networks – and supporting SOF. She understands cyberspace – the dangers as well as the great opportunity it provides for data discovery. She also understands how analytics can be used effectively to support the mission.

The Problem

As military operations shift and budget pressures shape the new military, supporting special operations forces when they deploy to austere environments, like Africa, with the speed and agility necessary will be essential. Developing a Common Operating Picture (COP) is challenging in the best of circumstances. How can we get the data that supports a COP in a place like Africa – and fast?

Some Solutions

There is a preconceived notion that there is no data available in the remote areas of Africa. But is that really true? “No!” say our panelists. We just need to start thinking out of the box and take advantage of what’s already there.

  • Open Source Data like population and tribal data, mineral data, water data, and lots more statistical global data is easily available
  • Non-Governmental Organizations (NGOs) running ongoing programs, such as Doctors without Borders and the Red Cross can provide a wealth of information.
  • OTTWs (Operations Other Than War) – oil and gas companies, mining companies, infrastructure and agribusiness companies, General Electric, Coca Cola. They almost never come into consideration as sources of information, but they have a lot to offer the military. They’ve already paved the way in this space and established their own infrastructure. They all have to have good situational awareness to provide for and protect their employees. They know who’s who and how to deal with local challenges. Is it unthinkable to ask them to partner?
  • Social Media Data – We know what you’re thinking, but hear us out. Much of Africa may be austere, but there are signs of digital life throughout. And wherever there’s a connection, somebody’s tweeting or snapping pictures with their cell phones and communicating information out in cyberspace on just about every human condition – illness, famine, harsh weather conditions. This social data may not be the kind the military traditionally uses to build a COP, but by connecting all the digital dots early warning signals can surface.

The underlying analytic technology keeps getting better and better. It’s more powerful and easier to use. Analysts can access and bring together big data from non-traditional sources. Unstructured data found on the web in in cyberspace can be incorporated into the analysis. Visual analytics lets non-statistical people quickly look at the data to spot trends and correlations. The end result is a more complete, holistic COP that can be built more quickly.

But, to effectively get access to the right data, the importance of partnerships can’t be understated. Trust and collaboration between data owners is imperative. The players supporting SOF in these austere environments need to reach out and build trusted relationships with NGOs, commercial industry and other government agencies. Players also need to build trusted relationships and share data amongst themselves. In this high stakes game, everyone is at risk.

Gail Bamford is an IDB guest blogger and senior marketing professional with over 25 years in information technology. She has been with SAS, the leader in advanced analytics software, since 2006 and supports business units focused on delivering analytic solutions to Defense, National Security, Higher Education and K-12.

Additional Resources:

Download white paper based on SAS-IDB Panel discussion at AUSA 2013, “Data vs. Gut” – “How Analytics Improves Decision Making at the Department of Defense: Finding new ways to add value and insights to big data.”

Read blog on SAS-IDB Panel at Indiana University, “SAS – IDB Defense Panel Serves Up Advice for Analytics Students at Kelley School Business Analytics Summit 2014”