Reference Checking for ERP selection

Updated: June 19, 2009

While there is an abundant number of white papers and articles on "How to Select an Enterprise Resource Planning (ERP) System", these white papers tend to focus on functionality requirements, technology issues and total cost of ownership. While these criteria are extremely important there are six other key elements that need to be addressed or your ERP implementation will probably fall short of your expectations.

Due diligence on the claims about the less tangible, but equally important elements of a vendor's total solution, can only be accomplished by asking other companies that have used or are using the ERP system being evaluated. In other words, reference checking is the only real source of truth. By not applying the same level of due diligence on references that was performed on functionality during the software demonstration process, some critical items will not surface until well into your implementation process and by then it will be too late to change.

Once the shortlist of ERP vendors has been identified, the due diligence process requires a dedicated and detailed effort to make contact with a vendor's current customers to construct a true picture of what you can expect to experience after your company becomes a customer. Most vendors have their two or three favorite reference accounts that they rely on for good reviews, so it's important to make sure and ask for a significant number of references. Experts suggest that you request lists of short-term customers (implementation in progress or within the last 6 months), mid-term customers (1 to 3 years after implementation) and long term customers (4 to 6 years after implementation.) If you only receive a few reference accounts it's a good idea to ask for references twice - first to identify the "pet" accounts (their input you may want to discount) and then for another set of customers who hopefully are consistent with the "pet" accounts. If there is inconsistent feedback from all the customers referenced then a deeper review is in order.

When checking references it's important to ask the right questions. Product functionality questions are mostly resolved during the software demonstration process, so the goal of checking references needs to focus on measuring the elements of a vendor's total solution that cannot be reliably validated by the vendor through presentations and software demonstration:

  1. Quality of the ERP Software
  2. Quality of the Vendor's Customer Support Organization
  3. Time, Pain and Cost of implementation and data migration
  4. On-going IT resources to support the ERP application
  5. Scalability of the ERP Software
  6. ERP Vendor's commitment to your success

Let's take a look at each of these elements:

  1. Quality of ERP Software

Obviously, high quality software is an essential requirement in an ERP system. The frequency and severity of software bugs is a critical issue that can impact a company on many levels, from the consistency of financial reporting to the reliability of serial number and lot tracking.

Software quality is an issue that cannot be resolved during the sales cycle in a demo or sales presentation. A sales engineer won't provide a true representation of a system's quality if they are having problems with a particular release or new product. Product specialists carefully script their demos to work around known issues so they do not come into play.

It's especially important to gauge the quality of systems that are new to the market or promote the fact that they are built on the latest technology platform. Sales reps will maintain that having the latest technology is a key advantage over a competitor's more established system, but in many instances new software systems and products developed on emerging technology are unproven and lack market support and real world validation.

Usability is a quality issue that most vendors will claim, but is hard to validate without sitting down using the software for an extended period of time in a prolonged real-world environment. One person's definition of "user-friendly" can vary wildly from another's based on the level of experience. If the system has bugs or shortcomings that require "work-a-rounds," users will be tasked with performing multiple steps to achieve a routine outcome and will quickly become frustrated with the software.

Investigate the genealogy of the software. Who authored it? How many owners or name changes has it undergone? Was it developed or acquired by this vendor? If the vendor is a reseller - do they have a position of strength with the owner / parent company if needed?

Gauging the quality level of a software system is an important consideration, and is an issue that can only be resolved outside the controlled sales environment. Talking to and asking questions of a good cross-section of individuals who are already using the system at their companies will be a good indication of the quality level of a system.

Questions to ask:

  • Is it reliable? Does it do what it is supposed to do?
  • Does the system lock up frequently?
  • How many bugs does the vendor say are in the system?
  • Are there any work-a-rounds that you have to do and how long have they been known issues?
  • Are there periods of time when you are required to log off the system to facilitate an MRP run or for End-of-Month processing?
  • How many software products does this vendor sell and support - are they focused on ERP.
  • If the vendor acquired the software:
    • When did the acquisition occur?
    • How many other platforms does the vendor sell/support?
      • If multiple platforms are being sold ask "why?" as this may result in customer support and/or development dilution. In addition, you need to fully understand if you will need to migrate up to a higher level platform as you reach your growth objectives.

  1. Quality of the Vendor's Customer Support Organization

Simple question; how much value-add can the ERP System deliver if you don't know how to use it correctly or if the implementation process is delayed or postponed?

The quality of the vendor's Customer Support organization will have a major impact on your company's ability to gain maximum value from the ERP system. In addition a strong training program is a key component of a vendors overall product offering.

Access to knowledgeable customer support representatives is a significant benefit that can reduce the likelihood of critical delays due to user error. Imagine you're trying to close the monthly financial records and the system is reporting huge manufacturing variances. Being able to call the support hotline and talk to an experienced professional who can calmly and logically walk through the most likely case scenarios with you to resolve the issue is an insurance policy that holds tremendous value.

The key answers that you need to determine are:

  • Can I easily get direct access to the Customer Support organization or I am I likely going to be stuck in a monolithic bureaucracy?
  • How knowledgeable are the Customer Support employees?
  • What is the response time for help or issue resolution?
  • How do the Customer Support employees treat their customers?
  • Will they "nickel and dime" us for every question asked?
  • Will we be dealing with a Reseller or will we have limited access to the ERP vendor?
  • Does the Customer Support organization really care about our problems or will we just be a number to them?

  1. Time, Pain and Cost of implementation and data migration

Every ERP sales person will tell you that their implementations will be done on time, with little pain and the professional service fees will be on-budget. Do you simply take them at their word or would you feel more secure if you asked some recently completed implementations about their experience?

A primary key to a successful fully utilized ERP system implementation is directly proportional to the planning done before the implementation is begun. Does this vendor offer a structured implementation plan - will you know What, When and Who will be responsible to execute each step of the implementation before you begin?

Some questions that might prove eye-opening:

  • Is there a written implementation and training plan with responsibilities, milestones and dates?
  • Did the implementation take longer than expected? If "yes", what was the main cause?
  • Were the consulting/professional service fees quoted enough to get the company "up and running" or were there overruns?
  • Was the quality of the training satisfactory?
  • How painful was the migration of the data?
  • Was the implementation team on-site or readily available during the first month-end close to ensure everything was going smooth?

  1. On-going IT resources to support the ERP application

Depending on the complexity of the database platform, the stability of the system and the usability of the logic and interface, IT resources needed to support an integrated ERP system may become a significant ongoing expense which may grow larger as your company reaches its revenue growth trajectory.

Systems built on reliable industry-standard technology are easier to support, and personnel with the skills to handle standard technology will be more plentiful in the marketplace. Upgrades will be easier to install which will ensure that internal resources can implement new releases of the software as opposed to hiring consultants.

Systems built on top of older technology will require system administrators with special skill sets. For instance, ERP systems that uses multiple databases - one supplying the foundation for the legacy core system and the other, newer database riding on top. Personnel familiar with both databases as well as the programming language of the middleware that integrates the two will need to be available to support the system.

Systems that are easier to learn and use will require less application support in training new users and answering day-to-day questions about process workflow.

Asking the right questions in the reference-checking process system can give you a good idea of what is required to support the technology of the system and avoid unforeseen IT expenses:

  • How many full time equivalents of IT people will be required to maintain and support the ERP application on a daily basis?
  • How many servers are required for the ERP application?
  • Are you able to perform upgrades to new releases on your own or does it require special consultants?

  1. Scalability of the ERP Software

Scalability is really a multi-part issue. The first being whether the breadth of functionality will support your company over the long term given your strategic objectives and growth projections. Secondly, you'll need to understand if the ERP system's performance will be hindered as the company's projected growth becomes a reality.

An ERP investment is a large expense, so it needs to be looked at as a long-term investment. Can the vendor's ERP software support me for the next two years, five years, or more than 10 years? Will additional hardware be required? Will we have to migrate to the ERP vendor's more robust platform or add modules as we grow?

To fully understand the scalability element it's essential to identify references in various stages of growth. The needs of a small vertically-integrated manufacturer will be different from those of an international company operating with a distributed supply chain.

Questions to references in this area are best pointed towards gauging the total cost of ownership of the system and identifying whether the software will be robust enough to expand with your growth trajectory:

  • How long have you been using the system?
  • How long do you expect the system to be able to meet your needs?
  • During your growth path, did you have to be upgraded to a more robust platform of the ERP vendor?

  1. ERP Vendor's commitment to your success

An ERP system is directly tied to a company's potential for success. The business logic provides structure and control which enables efficiency to be applied, measured and improved. Because a functional ERP system is so vital to the well-being of an organization, an ERP vendor becomes your business partner.

As your company grows, your needs will change and it's not possible to predict all the paths and opportunities that will be presented to the company. Will these changes require some modification to the ERP system or in the way in which you will be required to use the system?

To understand the level of partnership it's important to reference check how the ERP vendor goes beyond the "call of duty" to help its customers. Are people's efforts focused exclusively on generating more revenue, or offered because they sincerely care about the success of their customers.

Will the vendor be willing to marshal its forces to help you in times when opportunity knocks or when a crisis occurs? The time to determine this characteristic is before you make the final selection, because when you need to rally the troops, you'll want to be sure the resources will be made available.

Do you want to be one of many customers or be considered a special one where you are able to have a meaningful on-going business partner relationship?

To truly gauge the extent to which a vendor is likely to be a true business partner, try contacting a non-sales executive of the company. If you cannot get an executive of the company to talk to you about their company's business partnering resume before you are a customer, there's probably little chance they'll talk to you about your issues after you are a customer.

In summary, while system functionality may be a key determining factor of whether or not a product is a fit for your organization, the ERP selection process cannot end there. The intangible factors of the vendor's customer experiences after the sale hold the potential to severely impact the success of your implementation and require as much due diligence as the software demonstrations.

Only by talking to users of the vendor's software and asking the right questions can one truly get an idea of what they can expect after the sale and then make the right long-term decision.