Software Demos: How to Separate Hype from Reality

Updated: February 12, 2008

Issue


In the process of selecting new software, every IT manager will want to see a demo. But what should be a chance to kick the tires and put the software through its paces often turns out to be a game of smoke and mirrors, with vendors steering the discovery process. Here is how to cut through the hype of demos and make sure you get the software you need.

Strategies

First, start by developing a procedure for evaluating software demos. Begin with meetings of all stakeholders to create a list of needs, wants and concerns. Each feature desired in the new software should be specified in as much detail as possible, making it easier to draw distinctions between software packages on each task. For example, "order entry" is not detailed enough; you should also specify how the software will handle out-of-stock conditions. A wish list of desired features will help you avoid being sidetracked by features that may not necessarily further your mission.

The procedure for evaluating demos should also specify how vendors will be contacted, who will attend demos and what the protocol is for taking notes on each feature under evaluation. Detailed and consistent notes will make comparisons of software packages much easier later on in the process.

When you request a software demo, pay attention to what you get in return. Beware of vendors who send only "product tour" packages that merely paint pictures of what they want you to see. The same goes for limited software that does not allow you to fully test all features. The best software demo is a fully functional package — which may be time-limited — and includes full documentation. The documentation, after all, is a key part of the training and support of the software.

Evaluate the software's documentation, too. It demonstrates how easily the software can be installed and configured, how fast employees can learn to use it and how difficult it will be to customize the software to fit your workflow. The documentation may also list known bugs and work-arounds that you should know about, and it will give you a sense of how well the vendor supports the product.

A live, in-person demo conducted by a vendor's representative is often the best way to initially get your questions answered. If you have written down your questions beforehand, along with a protocol for evaluating the demo, you will have better luck interrupting the sales pitch to get the answers that really matter to you.

If the software will be deployed in a multiuser environment, it should be demonstrated under a simulated load of users similar to your own organization. A package that runs well with a light load of users may bog down under a full load. Likewise, the demo should include a set of data representative of your company's typical volume.

Demos generally present a best-case scenario from the vendor's point of view. Look for bugs in the software by giving it inconsistent or incongruous data to see how it handles such situations. For example, what happens when a sales order is input for a quantity of zero? How easy is it to edit data entries?

It is not uncommon for an IT manager to be invited to attend a free seminar that demonstrates a software package or to a technical meeting that discusses the software's inner workings. It is worth noting that many demo seminars are offered during the workday and tend to be mostly hype, while technical meetings are often held in the evenings and tend to be more substantive.

The best demos may not be given by vendors. Ask if the vendor has a user group (it should) and whether you can attend one of its meetings (you should be able to). User groups often put on hands-on demonstrations of installations, bug fixes and other issues that are not covered in vendor-provided demos. If you can gain access to a user group's mailing list where issues are discussed, so much the better. Talking to users can reveal much more about a software package than is shown in a demo.

A site visit to an existing customer can also provide a very realistic demo. If possible, ask the vendor for a reference to a customer similar to your company. Arrange a visit to see how the software is performing in actual use. This is a good time to ask the customer about the company's technical support, upgrade schedule and costs, and any other matters of concern.

Next Steps


Obviously, you won't have time to pursue user groups and site visits with every potential vendor. But once you have narrowed your candidate list to one or two strong contenders, the added effort to cut through the demo hype can pay off. The important thing is to avoid being a passive participant in a vendor's staged software demo. Take an active role to make sure the demo provides all the information you need to make a smart choice.

Featured Research
  • Best ERP Features and Benefits for Your Business

    Are you considering investing in ERP software for the first time? Or maybe you already have an ERP solution but you’re worried it’s becoming dated. If either of the above apply to you, read our latest guide on the top ERP features and benefits based on the size of your business. You may be surprised at how versatile and cost-effective it is becoming - regardless of if you own a small business or run a large enterprise. more

  • 9 Spooky Signs You Need a Contact Center Upgrade

    When was the last time you evaluated the performance of your current contact center and the software you are using? The results may be frightening! If it’s been awhile since you invested in contact center software, there is a good chance that your needs have changed or that there are better options available now. Fortunately, it’s relatively easy to determine if you need an upgrade or not. more

  • 7 Ways the Wrong Phone System Can Haunt Your Business

    The wrong phone system could be haunting your business - and we’re talking about problems more serious than ghosts and ghouls. From increased costs to issues with scaling, we’ve identified seven important ways that a less than ideal phone system could be holding you back. You’ll be surprised at how much of a difference this can make to your bottom line too. more

  • Ditch Your Fax Servers

    An in-house fax server gives an IT department centralized management and monitoring over the entire enterprise's faxing. This can help your company track usage and better maintain records for auditing and record keeping. However, there are serious drawbacks that come with utilizing an in-house fax server solution and these range from security to cost-prohibitive pricing. more

  • The IT Manager's Survival Guide

    As an IT manager, maintaining physical fax servers and infrastructure is not a high priority. However, fax capability remains a business need simply because chances are your industry is dependent on its security. What if there was a way to reduce the amount of time spent handling fax complaints and maintaining physical servers? And this way took into account security, cost savings, and freed up your IT resources. Would you be interested? more