When would you NOT set up a test system/sandbox?

In general...

If the system is heavily developed or configured for the company I would recommend a separate test system. If the system is a known quantity or has been in the market long enough, then most of the issues one may have can just be a result of implementation anxiety (which is normal). I.e. the providers usually know where the kink/problem is and easily corrected.

That being said, It also depends on how one perceives and know his/her pre-implementation plans are going. If one has a comprehensive (or a perception of it) plan and is going well (dotting the i's and crossing the t's) then i would recommend just going live. User Signoffs for every stage is important too.

I would recommend a comprehensive review of data migration (from the old system) and another round of review of the settings. I also recommend a Plan B (hang on to the old system) or C in case everything falls apart.

The vendor is incompetent.

Your never do major (and really shouldn't do) minor programming on a live system.

It is a recipe for disaster and contamination of data and programing should the scripts written fail.

I can't imagine not having a test environment where you can heavily test the configuration before going live. Since I have been doing system implementations with clients using our proprietary software for the last 24 years, I think I have the experience to back up that statement.

However, some types of testing should be done only in the live environment. After final configuration decisions have been made and tested, they should be set up in the live environment. And then they should be tested there as well, by doing parallel testing. I have found that doing parallels in a test environment can turn into a nightmare if you are trying to make sure any data or setup tweaks are also done in the live system.

I also agree with Emerson that you get strong user signoffs for every phase of the implementation. They should never be a pro-forma signoff. The user needs to be completely comfortable that their system operates as intended. I have actually refused to let a client sign off on occasion if I felt that they really did not understand what they were signing - even if it delayed the go-live.

Doing a major ERP conversion without adequate planning/testing is an invitation for disaster. I worked with a major hospital system that apparently tried to shortcut testing, went live on their new system, and then discovered they couldn't write vendor checks ... for several weeks!

I have implemented a cloud based software solution into a live system where any transactions will be clearly marked as originating from the software, volume of transactions will be very low by limiting scope initially and transactions can be reversed or deleted if errors occur. I agreed to do this because we would lose weeks of programming time and incur additional costs by running on our test environment. It really becomes a cost / risk / benefit analysis.

Well said everyone. A test environment doesn’t take time – it saves time. Almost every programming change, no matter how small, goes through more than one iteration. If the data becomes corrupted along the way, programmers will inevitably be chasing after bugs that don’t exist. A test environment also allows users to try things they would never risk on live data.

From the vendor side of the fence I would always insist on a test system where everything is vetted out and where all future changes can also be tested when required.

To answer the question, I would NEVER - EVER - NEVER agree to install anything without testing, except on my own system.

That is because I am willing to take full responsibility for anything that goes wrong. I can control everything, so, I can be wary of what I do to cause problems.

With anyone else, I am NOT doing anything without testing. Too many variables. Plus, you have a person who will focus responsibility on me if anything goes wrong, even if it was caused by the Trojan in the latest cat video they watched.

When things go wrong, depending on the severity of the problem(s), someone's job could be on the line. When jobs are at risk, everyone clams up and has amnesia about what they did, covering up any evidence while stonewalling.

Though computers can hold a lot of records, those records can disappear in an instant. Those records that would have protected you, and pointed the responsibility in the right direction.

This is how things should work:
1. All responsible parties involved should agree to sign off at their stage of the process.
a. Explain that signing off does not mean they are accepting that the previous stage is working 100%.
b. They are signing off that everything expected in their stage of implementation is complete and working. (100%. If part of their responsibility is done and must be progressed while another part is not ready, before progressing anything, split up the two stages and follow them separately.)
c. DO NOT move ahead on any stage until the previous stage is signed off. Refuse to allow moving ahead, even if your job is on the line, because if you allow progression of a project and it fails, your job IS on the line.

2. If someone forces progression before signoffs, then require them to take responsibility, along with recorded evidence that you are moving the project forward under protest. If they question you about the paper trail, explain to them that by having a record, you can prepare to respond quickly should anything go wrong. By saying that, you will either get them to wait, or will alleviate fears that you are making a paper trail to cover yourself and point responsibility on them.

Now, here's how things should develop:
1. Determine specs for the project
2. Design project to specs
3. Build project
4. Test project
a. While testing, begin training your SMEs
5. Implement project
6. Monitor project
7. Retire in the Bahamas or Hawaii

The Test cycle is this:
1. Run Test
2. Evaluate results
3. If test passes, begin implementation. If test fails, redesign and build and do Step 1 again.

Repeat until all tests are passed.

Implementation goes easy and quickly in a ratio equivalent to the level of adherence to testing...that is, the more thorough the testing, the easier the implementation.

Now, about ready-made products. Would I install Microsoft Office without testing?

Sure. But not because I ignored the previous rules laid out. Rather, it is because Microsoft did the testing for me. They have invested a lot of money to make sure their product is stable, and functions exactly as expected.

Packaged products do not eliminate the need for testing. Instead you are paying the manufacturer for already doing the testing.

What Wayne said. I assume an "Eecipe" is his code term for "Extreme Recipe" for disaster and contamination. Is there another industry that submits product to market and/or end user without testing? That would be madness right? Why is software different?

Test, test, and test again.

Thanks all, I appreciate your comments (with plenty of implicit warnings about risks!!), which makes me wonder why people take these risks time and time again.

End users may assume (and they may be trusting or naive) that their implementation partner will LEAD them with sound practices and tools to succeed. If that trust is misplaced, if the end users don't understand what they really need to do, then a failed implementation is highly likely.

A development sandbox, always. We even have an intermiediate "quality testing" environment between "dev" and production ["prod"[.