How DoD Supply Chains Can (and Should) Learn from Private Sector Mistakes:

Using a “Value Added” Approach in Supply Chain Forecasting

By IDB Guest Blogger: Michael Gilliland, SAS


Materiel and supplies are critical to the men and women of the U.S. Armed Forces, at home and abroad. Whether it’s ammunition, weapons, IT, or toilet paper, if they need it, someone is in charge of getting it and making sure there is sufficient inventory. The Government Accountability Office has identified this mission-critical role – supply chain management – as an area for improvement for the U.S. Department of Defense (DoD), particularly inventory management through better forecasting. As DoD works to improve its supply chain forecasting capabilities, this is an area where DoD can learn from the mistakes of the private sector.

The reality is that forecasting can be a huge waste of management time. This is not an indictment of the practice of forecasting as a whole, but rather of how organizations usually approach and apply forecasting incorrectly.

The problem is not that forecasting is pointless, irrelevant, or unnecessary. Rather, the problem is supply chain leaders squandering too much time and too many resources on forecasting with a myriad of bad practices.

Tools and Methods

In some cases, the technologies organizations use for demand planning and forecasting are outdated or simply misapplied. By relying on outdated tools or methodologies, these forecasters miss the progress made in recent years to improve accuracy, reduce bias, and minimize the cost of forecasting through large-scale automation. ​ Even heroic efforts on their part are likely to deliver underwhelming results, whereas an unpoliticized and unbiased forecast can lead to cost savings and, more important, can save lives.

Expectation Management

The goal of forecasting is to obtain an objective, dispassionate number that is as accurate as can reasonably be expected given the nature of whatever you happen to be forecasting. Rather than working from this perspective, however, many managers and forecasters have unrealistic expectations for the level of accuracy achievable. They rely too heavily on the current “fit” of models to history when their job is to forecast the future. Almost invariably, the forecast will be less (often much less) accurate than the fit to history. (It is always much easier to explain the past than to predict the future.)

 Don’t Trust the Process

Perhaps the most blatant example of waste in the forecasting process is “forecasting by committee.” This is where a forecast is passed through so many different stages of approval and has been tweaked by so many collaborators that its integrity is actually degraded. It is easy to see how this approach could occur in any bureaucratic environment – including the military – where each participant has a personal agenda they express with their adjustment.

The problem is that this kind of elaborate review process ends up being extremely costly – in two ways. First, because the process is unnecessarily consuming everyone’s time. And second, because your outfit may actually be in a worse position than if you had not attempted to incorporate so much “management intelligence” into the forecast in the first place. In a study of eight commercial supply chain companies, Steve Morlidge (author of the book Future Ready) found less than half their forecasts were more accurate than the “naïve forecast” – i.e., the forecast you get by doing nothing and simply using the last available data point as your future prediction.

Know Your Limits

Even if you use the proper tools and methods to forecast future supply chain needs, it is important to realize that overall forecastability still limits the maximum possible accuracy of forecasts. Supply chain leaders need to avoid demanding a level of forecast accuracy that is simply impossible to obtain because they have not considered the nature of what they are trying to forecast.

To illustrate this concept, suppose your job is to forecast Heads or Tails each day in the tossing of a fair coin. While you may have some lucky streaks and forecast correctly several days in a row, over the long haul your forecast will be correct just 50% of the time. It doesn’t matter if your ranking officer demands 60 percent accuracy or higher – you are limited to a 50 percent accuracy ceiling by the nature of the behavior you are trying to forecast. Not even bigger computers and more sophisticated software will help – there is nothing anyone can do to achieve 60% accuracy. Unachievable objectives motivate forecasters to simply give up, or find a way to cheat and game the system.

supply chain forecasting scatterplot

Volatility and Forecast Accuracy. Scatterplot: Mike Gilliland.

As seen in this scatterplot of 5000 items being forecast by a consumer goods manufacturer, the variability or “volatility” of a demand pattern has a big impact on how well we can expect to forecast it. Smooth, stable, repeating patterns can be forecast quite accurately with simple methods. But wild, volatile, erratic patterns may never be forecast accurately, no matter how many resources we commit to forecast them. To the extent that we can control and limit volatility, we are likely to achieve more accurate forecasts.

Solution: Knowledge is Power

Private sector companies are increasingly using a method called “Forecast Value Added (FVA) Analysis” to improve performance of their forecasting processes. FVA is the application of basic science to evaluate a forecasting process – measuring each step in the process, and determining whether it is “adding value” by making the forecast more accurate and less biased. Here is an example of an FVA report (in this case, the Analyst Override step is just making the forecast worse!):

supply chain forecasting: FVA example table

Example: Forecast Value Added (FVA) Analysis. Table: Mike Gilliland.

Forecast Value Added analysis is about rooting out the waste and inefficiency from forecasting efforts (so is consistent with a lean approach to supply chain management). It allows organizations to streamline their process and redirect the non-value-adding efforts into more productive activities that will be more beneficial to the mission at hand. Understanding cautionary tales of corporate sector mistakes and challenges can help the DoD create real improvements in defense supply chain management. By avoiding wasteful steps and procedures, defense organizations have the opportunity to achieve better forecasts with less effort, and less cost.

Gilliland, Mike headshotMichael Gilliland is an IDB guest blogger with more than 15 years of forecasting and supply chain management experience in the food, apparel, and consumer electronics industries. Gilliland has been featured in the LOGTECH Advanced Program in Logistics and Technology. He is author of The Business Forecasting Deal, and has published articles in Supply Chain Management Review, Foresight: The International Journal of Applied Forecasting, Journal of Business Forecasting, Analytics, and APICS magazine. He is also an editor for Foresight. Gilliland holds a BA in philosophy from Michigan State University and master’s degrees in philosophy and mathematical sciences from Johns Hopkins University. He writes The Business Forecasting Deal Blog at For more than a decade, SAS and the IDB have worked collaboratively to raise the level of awareness of the value analytics to defense leaders.

Students Get a Taste of Government Analytics at IDB Hosted Panel

By IDB Guest Blogger: Gail Bamford, SAS

When Tom Davenport and D.J. Patil suggested in their article published in Harvard Business Review that the data scientist is the “sexiest profession of the 21st century,” these in-demand professionals became part of the discussion surrounding the big data ground swell.

On April 10, 2015 at the 3rd Annual Business Analytics Forum at Indiana University’s Kelley School of Business the Institute of Defense and Business (IDB) and SAS had an opportunity to address over two hundred students, faculty members and industry partners who have a vested interest in grooming these superstars of the future.

The Kelley School’s Institute for Business Analytics , co-chaired by Vijay Khatri and Frank Acito, was one of the first programs of its kind established to prepare students for careers in business analytics. The Institute hosts this conference to make their students savvy in real world analytics and bridge the gap between academia and the real world. They actively seek out and engage with industry partners who provide internships and other pathways to employment.

This year’s conference topics included the Internet of Things (IOT), supply chain analytics, and healthcare analytics. The IDB, SAS, Deloitte and IBM were invited to inject the importance of analytics to government.

sas infographic government analyticsGovernment Analytics Jobs

Government is a heavy user of analytics and acutely feels the pain of the analytics skills gap across all agencies. A 2014 GovLoop survey reported that 96 percent of those surveyed – 46 percent of whom self-identify as experts or analysts – believe their agency has a data skills gap

Last year’s audience gave the thumbs up to our “Data vs. Gut” panel discussion, which featured retired military officers giving their perspective on the evolution they’ve seen of analytics use in decision making in defense organizations. This year, due to unforeseen circumstances, our defense panelist was not able to make it. Van Noah, panel moderator, smoothly shifted gears and reworked the flow of the conversation to provide the students in the audience with a broad preview of how they can apply their data skills in government analytics jobs.

If you were not able to see the discussion in person, here is a summary of the content these experts were able to shed light on:

  • Since this panel discussion was held just five days before the income tax filing deadline of April 15th – and stories about scammers were prolific in the media — what better subject to open the conversation?
  • Indiana University Kelley School of Business 04.10.2015Van Noah from IDB was able to represent the government sector and weave in some thoughts on how the military could better use analytics to select candidates for intensive training programs. For example, pilot training is expensive, so using analytics to identify students who are likely to wash out early is much safer and more cost effective than letting all students progress to the next phase of training.
  • Satish Lalchand from Deloitte has a long history of working with analytics in government and provided examples of how the federal government is leveraging analytics to combat fraud and improper payments. He also shared his perspective on how agencies can get started once they see the business need for analytics and what it takes besides analytics to make better decisions.
  • Eric Zidenberg from SAS has been involved with public safety organizations for many years and talked about how analytics are currently being used on the southwest border to make better decisions on which cars should be sent secondary inspection for illicit materials.
  • Dion Rudnicki from IBM segued nicely into talking about how the Memphis Police Department was using analytics to better allocate resources to reduce crime. In a city previously identified as #1 for crime in the US, and where there are only 2,500 police officers to protect the population of 650,000, analytics allows government to place the right resources at the right places at the right time. The result: a 30% drop in crime.

Next Generation Data Scientists

Students preparing for careers in data science have a lot of options when they enter the workforce. The data analytics talent gap exists in every sector in our global economy. When students weigh their options, they should consider the jobs that support the public sector. Serving government will give them a chance to make a real difference. Government needs these talented, data scientists of the future!

Gail Bamford is an IDB guest blogger and has over 30 years of experience working in the public sector IT market. She has been with SAS since 2006 and is passionate about helping close the analytics talent gap. For more than a decade, SAS and the IDB have worked collaboratively to raise the level of awareness of the value analytics to defense leaders.

Data, Gut, Watson, and Angelina Jolie

The fourth and last of the Data vs. Gut panel series took place at the 2014 North Carolina Federal Advanced Technologies Review (NCFATR) conference on June 5. Scott Dorney, Executive Director of the North Carolina Military Business Center invited the Institute for Defense and Business (IDB) and SAS to give a repeat performance of the original Data vs. Gut panel discussion last fall.

The discussion was moderated by IDB president, Mark Cramer, and featured Karen Terrell, VP of SAS Federal and former IDB Executive Fellow, Rear Admiral Erroll Brown (USCG ret.).June 5, 2014 NCFATR Panel "Data vs. Gut"


In every panel discussion we’ve hosted on Data vs. Gut, the importance of data leads the discussion. Good data is critical to successful analytics. It’s not surprising that this comes up over and over since there are many examples of bad data leading to disastrous decisions with disastrous consequences. In doing analysis, be mindful of:

  • Data quality: How accurate is your data? Accurate data analysis is only as good as the data put in. A lot of data is still entered manually and is incomplete or incorrect.
  • Timeliness: How important is timely data in your analysis? Time-critical decisions depend on easy access to the most up-to-date data.
  • Data relevancy: What data do you need to solve your problem? You don’t have to analyze every bit of data you have access to. What data is relevant to your situation?

Brown had some observations and suggestions.

  • Problem: Clearly understand the problem you are trying to solve.
  • Data: Is the data right and is it the right data? Make sure you know your data. Make sure it is accurate and timely. And, make sure the data you are using in your analysis is relevant to the problem you are trying to solve.
  • Tools: Get the right analytic tools for the job. One size does not fit all. You have to have the right algorithms, or “algos” too.

Evolution of Data and Analytics

June 5, 2014 NCFATR Panel "Data vs. Gut"Terrell shared with the audience her observation of how the usage and value of data has evolved. In her 20+ years of serving the government market, she has seen customers move from basic reporting and descriptive statistics to advanced and predictive analytics. Government routinely uses analytics to detect and prevent fraud these days. Other areas trending up are predictive asset maintenance and identification of high risk procurements.

She also noted the “democratization” of analytics. If you know your data and know your business, you can unlock insights in your data with powerful visualization tools that make it possible to slice and dice your data from your desktop. It’s easy and you don’t have to be a statistician to understand the graphic output.

High performance analytics is allowing more data to be analyzed faster and with a greater degree of granularity. Terrell mentioned a large department store chain that is using analytics to do markdown optimization on 270 million SKUs in 850 stores. Optimization models determine which items to mark down and by how much. What used to take 30 hours to run, now takes less than 2. This saves the company millions of dollars every year in unnecessary markdowns.

Data vs. Gut

Terrell brought the data vs. gut discussion to a very human level when she mentioned Angelina Jolie, who saw both her mother and aunt die at early ages from cancer. In her NY Times Op Ed piece she talks about the reality and her decision. But, her radical decision was not made in a vacuum. She did her research and gathered the data. She carries the “faulty” BRCA1 gene and found out she had an 87% risk of developing breast cancer. With this knowledge, she was able to make the difficult, but data-driven decision to have a preventative double mastectomy.

June 5, 2014 NCFATR Panel "Data vs. Gut"Brown cautioned about machine-to-machine decision making. “Computers don’t make you smarter.” That’s where gut comes in. Humans must have a symbiotic relationship with machines. But, Brown, who spent several years at IBM after his retirement from the Coast Guard, couldn’t resist bringing Watson into the conversation. Watson learns like an individual and can tell you how it got to an answer or outcome. That’s more than he says he can get out of his seventeen-year old when he has confronted him with “How’d you get to that (stupid) decision?”

So if Watson processes data more like a human than a computer and learns as it goes, what will the data vs. gut discussion be in 10 years?

Gail Bamford is an IDB guest blogger and senior marketing professional with over 25 years in information technology. She has been with SAS, the leader in advanced analytics software, since 2006 and supports business units focused on delivering analytic solutions to Defense, National Security, Higher Education and K-12.

Additional Resources:

Download white paper based on Data vs. Gut panel #1 – “How Analytics Improves Decision Making at the Department of Defense: Finding new ways to add value and insights to big data.”

Read Blog on Data vs. Gut panel #2 – “SAS – IDB Defense Panel Serves Up Advice for Analytics Students at Kelley School Business Analytics Summit 2014”

Read Blog on Data vs. Gut panel #3 – “Shining Light on Data in Austere Environments”

Big Data for National Security

A few weeks ago, during the Special Operations Forces Industry Conference, the Institute for Defense & Business in conjunction with SAS Institute sponsored a luncheon panel discussion about Analytics and the Future of SOF. They offered me the opportunity to partner with Ambassador (retired) David Litt and Major General (retired) Kevin Leonard, U.S. Army to discuss how analytics can be used as an enabler for SOF operations in austere areas like Africa.

We’ve all heard the hype about “Big Data.” But SO WHAT? It’s generally discussed in the context of marketing, with one of the favored examples being Target identifying which of their customers are pregnant through their buying habits so that they can then send them coupons for diapers. But if you are not trying to sell diapers, you have probably missed out on the conversation, since selling diapers is a far cry from national security.

So instead of discussing “big data,” why don’t we discuss “Predictive Battlespace Awareness” or “Intelligence Preparation of the Operational Space”? These are two topics that are highly relevant to National Security. Funnily enough, both of these discussions lead us right back to big data, analytics, and the subject of last week’s conference, how can analytics be an enabler for SOF in the future.

Predictive Battlespace Awareness is the concept that you can develop a situational awareness picture so rich it enables you to anticipate what will happen so that you can develop course of action to thwart your adversary before they can act. A recent blog on my website discusses agribusiness in Africa as a sample use case. Achieving situational awareness is one element of the decision cycle: Observe, Orient, Decide, Act, also known as the OODA-loop.

For years, in all industries, the defense community not withstanding, we’ve been making huge investments into sensor technology to enable the observation phase of our OODA loop. And now we find ourselves overwhelmed with that very same data. We are no longer looking for a needle in a haystack, we are looking for a specific needle in a needlestack, and, as one might expect, it’s pretty painful.

Analytics technologies allow us to explore that data in a way that empowers us to advance our decision cycle through the orientation and decision phases. This means we can act more quickly when opportunities present themselves, respond more rapidly to thwart our adversaries, and anticipate where problems might develop so we can put remedial measures in place ahead of time.

Further, big data analytics capabilities can act as an enabler to collaboration. In an age of complexity, the next operation may require cooperation across multiple nations, non-governmental organizations, and both civilian and military authorities. Big data can facilitate this, even despite the fact that each organization runs on their own siloed system and processes.

In each of these cases, Special Forces will be able to leverage huge amounts of data to develop an intelligence picture of their operating space, collaborate with a coalition of any size and shape, and build the network they need to manage the complexity that is a part of normal operations.

This essentially recaps the discussion between David, Kevin at the discussion panel. Once again, I’d like to thank SAS and the IDB for the opportunity to share ideas. Big data may have us swimming in the ocean, but analytics capabilities offer a huge return on investment, not just for people looking to sell diapers, but for those of you who work in the national security community—in fact, the return on investment from a national security perspective is infinite!

Lt Col Tammy Schwartz, USAF (Ret.)Lt Col Tammy Schwartz, USAF (Ret.) is an IDB guest blogger and recognized innovator with more than 20 years of national security experience. Schwartz is a former Chief Technology Officer for Air Force Enterprise Networking and is currently a Consultant for SAS and the Owner of Llamrai Enterprises. You can read more from Schwartz on her blog.

Shining a Light on Data in Austere Environments

SAS and the Institute for Defense and Business (IDB) co-hosted their third panel discussion May 21st during the Special Operations Forces Industry Conference 2014. Retired Ambassador David Litt, head of the Center for Stabilization and Economic Reconstruction (CSER) program at the IDB, moderated the lunchtime discussion entitled, “Supporting SOF Network in Austere Environments: Using Analytics to Prepare, Sustain and Support.” In this discussion, the “austere environment” meant Africa.

The panelists, MG Kevin Leonard, USA (ret.), and Lt Col Tamara Schwartz, USAF (ret.), had interesting perspectives given their different backgrounds, but they reached similar conclusions. While technology, data, and analytics are enablers, it is TRUST, PARTNERSHIP, and COLLABORATION that make it work. The challenge is busting old paradigms and thinking outside the box.

From left to right: AMB (Ret.) David Litt (CSER), MG (Ret.) Kevin Leonard (Fluor), and Lt Col (Ret.) Tamara Schwartz (SAS Consultant) discuss data in austere environments during the May 21st panel sponsored by IDB and SAS

From left to right: AMB (Ret.) David Litt (CSER), MG (Ret.) Kevin Leonard (Fluor), and Lt Col (Ret.) Tamara Schwartz (SAS Consultant) discuss data in austere environments during the May 21st panel sponsored by IDB and SAS. (Institute for Defense and Business Photo by Christine Reynolds)

The Panelists

Leonard knows AUSTERITY and LOGISTICS. During his long career with the Army, he’s been involved in logistics activities in Afghanistan, Jordan, and Iraq – just to mention a few countries. He has supported BIG ARMY and the little Army and is continuing to support contingency operations at Fluor. He knows the challenges of getting supplies over the last mile. He also knows that if Coca Cola can do it, it can be done.

Schwartz is well acquainted with IT and enterprise networks – and supporting SOF. She understands cyberspace – the dangers as well as the great opportunity it provides for data discovery. She also understands how analytics can be used effectively to support the mission.

The Problem

As military operations shift and budget pressures shape the new military, supporting special operations forces when they deploy to austere environments, like Africa, with the speed and agility necessary will be essential. Developing a Common Operating Picture (COP) is challenging in the best of circumstances. How can we get the data that supports a COP in a place like Africa – and fast?

Some Solutions

There is a preconceived notion that there is no data available in the remote areas of Africa. But is that really true? “No!” say our panelists. We just need to start thinking out of the box and take advantage of what’s already there.

  • Open Source Data like population and tribal data, mineral data, water data, and lots more statistical global data is easily available
  • Non-Governmental Organizations (NGOs) running ongoing programs, such as Doctors without Borders and the Red Cross can provide a wealth of information.
  • OTTWs (Operations Other Than War) – oil and gas companies, mining companies, infrastructure and agribusiness companies, General Electric, Coca Cola. They almost never come into consideration as sources of information, but they have a lot to offer the military. They’ve already paved the way in this space and established their own infrastructure. They all have to have good situational awareness to provide for and protect their employees. They know who’s who and how to deal with local challenges. Is it unthinkable to ask them to partner?
  • Social Media Data – We know what you’re thinking, but hear us out. Much of Africa may be austere, but there are signs of digital life throughout. And wherever there’s a connection, somebody’s tweeting or snapping pictures with their cell phones and communicating information out in cyberspace on just about every human condition – illness, famine, harsh weather conditions. This social data may not be the kind the military traditionally uses to build a COP, but by connecting all the digital dots early warning signals can surface.

The underlying analytic technology keeps getting better and better. It’s more powerful and easier to use. Analysts can access and bring together big data from non-traditional sources. Unstructured data found on the web in in cyberspace can be incorporated into the analysis. Visual analytics lets non-statistical people quickly look at the data to spot trends and correlations. The end result is a more complete, holistic COP that can be built more quickly.

But, to effectively get access to the right data, the importance of partnerships can’t be understated. Trust and collaboration between data owners is imperative. The players supporting SOF in these austere environments need to reach out and build trusted relationships with NGOs, commercial industry and other government agencies. Players also need to build trusted relationships and share data amongst themselves. In this high stakes game, everyone is at risk.

Gail Bamford is an IDB guest blogger and senior marketing professional with over 25 years in information technology. She has been with SAS, the leader in advanced analytics software, since 2006 and supports business units focused on delivering analytic solutions to Defense, National Security, Higher Education and K-12.

Additional Resources:

Download white paper based on SAS-IDB Panel discussion at AUSA 2013, “Data vs. Gut” – “How Analytics Improves Decision Making at the Department of Defense: Finding new ways to add value and insights to big data.”

Read blog on SAS-IDB Panel at Indiana University, “SAS – IDB Defense Panel Serves Up Advice for Analytics Students at Kelley School Business Analytics Summit 2014”