It's the process stupid!

One thing that everyone says about SAP - the processes can be long winded. However, you have to understand the reason behind that. The software was developed for use by very big companies and to meet the requirements of very restrictive legislation. As a result, many of the business processes seem to require more work than you would find in other software packages.

As I work primarily on the admin side, it's worth mentioning that the process of applying user access is particularly long winded. But there is very good reason for this - when you have 10,000 plus users, it is absolutely crucial to give all of these people the appropriate permissions. But working out what is correct can take a long time. And it has to be said that this has caused some friction amongst our project team.

Our basic AD permissions have been set-up over many years. They are not perfect by any means, but generally people have the necessary access and are kept out of those areas that they should not see. If we have to make changes, it usually only takes a few minutes and it is a pretty straight forward process that can be carried out by anyone in the IT team at any time during the day.

Unfortunately SAP is not so simple. A few months ago, I found out about Central User Administration (CUA), and we wll be looking to implement this at some stage, but at present we don't use it as it was not set-up at the beginning of the project. User admin requires details to be maintained in each system which is awkward and time consuming. The actual access permission is based upon a series of roles - permissions for transactions and authorisation objects are added to the role, and then user is placed in the role. A series of "transports" allows the roles to be copied between systems, so these should be the same in each of the systems.

However, the process is also designed to allow the change(s) to be made and then for these changes to be checked - first in the development system, then the test system, before it finally arrives in the production system. This is to ensure that any such changes are appropriate and don't do something that they shouldn't do. It makes a lot of sense and for the larger companies, I can see that this would be absolutely essential. For us, it is a very tedious process and we are not doing it as we should.

Unfortunately, even tho' the project team have been told about the correct process (repeatedly), they just don't get it. I've noted many times that I have been asked to allow a particular person to have a particular access permission - when I apply it, I've indicated that this then allows someone else within that role the same permission. We could have people able to do specific tasks that we would rather they didn't. After the first few months, I set-up a process to try to get these changes authorised properly. It sort of works, but even now people will try to bypass the process.

One other thing that I've noted - when we started, we set-up the roles based upon the job role descriptions that we use within the business. This was the advice from the consultants and I agreed that it made the most sense. However, having been doing it for some time now, I feel that we might need to look at this again as we have a lot of overlap in what various people do - the number of staff is limited, so most people actually do 2 or 3 "jobs". There is a good argument for changing some of the permissions so that instead of being applied to a "job role", we might have a "process role".

For example, we have sale clerks with various transactions, but there are a couple of these transactions that might also be requested for other people not in sales for various valid reasons - they could be used by accounts, by shipping, by purchasing, by production, as well as a couple of others. This is in fact, 15 roles in total. At the moment, these permissions have to be added to each of those roles, and after testing, they have the same permissions - potentially, we could have just 2 roles with those permissions, and then apply people to them in order to do the work and it would work just the same. Obviously, more work to set-up to begin with, but going forward, possibly much better as it would require less admin work.

Oh well, another day, another dollar!

Almost lost for words

I don't like to hear people swear. You hear it too often - so called "celebrities", sports or music stars using language that would make a drill sergeant blush. Often, the argument is that they are just passionate about the specific view they are putting forward, but I feel that in reality, it shows a lack of intellect. They swear because they don't know how to convey a point of view. But despite my views, this last few days, I have done more than my fair share of swearing.

Let's go back to the beginning. Some 2 1/2 years ago, the consultants started putting in the SAP system. All they did was install the basic software and addons - they didn't apply any of the patches or updates. At the end of last year, they made a comment that it was for us to perform these updates. It took me a while to get the information that I needed, but in the early part of this year, I managed to carry out a process, to apply the patches that they had downloaded, but not actually put in place.

Subsequently, I went onto the SAP Service Market Place Download Center and found that there were a great many other patches that they had not downloaded. I started work on downloading these, but after I reached a certain point, a message appeared to say that all patches released after a certain date had to be confirmed through SAP Solution Manage before they could actually be downloaded. As they had not installed a SAP Solution Manager system, this caused some problems - however, I was able to get around this eventually. As SAP now require Solution Manager to be used, I felt that we should get it installed - we have a copy of the software and the existing license allows us to use it, so I saw no problem.

I got a copy of the relevant SAPpress book and downloaded some material on the product and its use - it became clear that Solution Manager offers a lot of other functionality that we would find very useful and from conversations, it seems that SAP are going to increase the requirements for running the product. So we've spent a little time in between other jobs, getting the software up and running.

Having finished this over the weekend before last, I decided on Monday to take a look thru the product. It runs thru the SAPGUI and looks much like the main ERP but with a few different menu items. I glanced at a few of these, not really going too deep into the various areas - but then I saw something that caught my eye. When I opened part of the menu item for Solution Manager, I saw the phrase "ASAP Focus".

Those of you that have been following me for a while will know that I had a brief exchange with Michael Doane (http://sapsearchlight.blogspot.com/2009/08/sap-implementation-projects-still-crazy.html), a consultant with many years experience of working with SAP. He described a methodology to carry out an SAP standard implementation called "ASAP" which he advised should be used by all system integrators (consultants). I had not heard of this and it appeared our consultants didn't use it. Michael was a bit frustrated (and I could see why) as this is supposed to be used by everyone.

And here it was - a full description of the processes, documents to assist, presentations, step by step guides, glossary, checklists, issue registers, and a complete roadmap for the whole implementation project. Basically everything that we could have used and would have found useful - instead we had to develop our own systems. The more that I looked at this, the more frustrated I became, and of course, I then started to really let rip with the bad language.

So now we are starting to look at getting the Soution Manager installed properly. We will have to get someone in that knows about it, and get one of our staff trained up. It's just a pity that it has taken so long to find out about the product. It would be interesting to see what the senior management from the consultants has to say on why they didn't use it, but I dare say that I will not be allowed to ask the question.

Belated happy thanksgiving everyone.

Where does the time go?

I've just realised that it is nearly a month since my last update and about 7 weeks to the end of the year. Where does the time go?

We are still struggling with the SAP system, and now the consultants have all gone so we are very much on our own. We have a list of items that they still haven't gotten working, and our CEO spent over a hour on the phone to their director to try to get them to accept responsibility to fix these items, some of which have been on the issues list for 2 years.

I managed to resolve part of the problem with the invoicing - we now get the invoice run every day. At a meeting earlier in the week, our project manager said that this had totally fixed the problem, but after the meeting, he admitted that about half of the invoices are still not coming out right for one reason or another. Our finance manager did express some concern that without invoices going out, we won't have money coming in, and this could cause serious issues - we could potentially run out of liquid cash by the end of 1st quarter next year if the situation doesn't change.

In addition, it appears that there are a lot of jobs that haven't been invoiced at all - one of the guys did a quick check and it appears that this is actually an internal problem. People are completing a task, but it's not getting marked as complete, so the system won't process the job. Unfortunately, we can't just force it thru as there is no easy way to tell if the work has been done or not - it generally takes a couple of hours to confirm, by searching thru various paperwork.

There is still an issue with purchasing - jobs get re-scheduled, but the due date for items is not being adjusted correctly and several times, we found ouselves waiting for goods, or items were being delivered for something that has been put back acouple of months. It's more of an annoyance than a serious problem, but as part of the overall project doesn't make it look good.

The system is still not giving us many of the reports that we need. Various people were told to use different transactions, variants of specific reports etc. but many of these don't really seem to give us what we really need. Some of them seem to provide vey large amounts of data that is actually of no value what so ever. I had hoped to be able to spend some more on this during the project, but just never seemed to have the time.

As a result, I've booked one of my staff onto an SAP training course - it's way across the country at one of their actual training centers, and we want to see if the courses actually help; if they do, we may book some more. We do need to get better information about using the system, and we don't want to have to keep paying huge amounts every time we want to make minor changes. The courses are expensive, but if we get our people better trained, then it will prove worthwhile - having invested so much, we have to make the best of it.

Have to go, as we're off to visit my wife's parents - they're going on a cruise next week. Never thought that I would be envious of them!

Oh boy!

I definitely spoke too soon... the question is where to start?

On Monday, there was a meeting about some of the deliveries. It appears that there is a big problem with some that have been sent out on the wrong dates. After considerable analysis, it appears that one of the processes used to check availability of products is re-setting any date that has been entered for delivery, and making it due 7 days after the inquiry. So a load of orders have been rushed thru production and despatched - but in a number of cases, they have arrived well before the actual date the customer wants.

In some instances, they have hung onto the products, but have asked that we accept delayed payment, and to keep them happy, we have had to agree to that. In a couple of cases, the product has been refused and we will have to re-deliver later on (in some cases, next year). Meanwhile, production are upset because they have had to do rush items and it has caused some scheduling issues.

To fix the immediate problem, the Sales Manager sat down with all orders and has gone back over each item and re-entered the required date. However, after his first attempt, one of his staff had gone back in to check the availability using the same process and once more, it had reset dates. As you might imagine, he was pretty pissed about it. They are now checking dates on all outstanding orders each day to try make sure that it doesn't happen again.

We also have a major issue with some stuff that was marked for export. The guy responsible asked for the items to be put on the system, but unfortunately, the overseas client had already placed the order - although the system is supposed to flag duplicate purchase order numbers from the customer, it still isn't doing so as I mentioned last week.

The goods were made twice and as they are non-standard items, we are not going to be able to sell to anyone else. It's cost us about $10,000 not including shipping. To make sure it doesn't happen again, they have developed a manual system of checking each order - at one point they asked if I could create an Access database to monitor the orders!

Finance are not doing well either. A consultant had set up a process for running invoicing automatically - but so far, it has only produced about 1 invoice in 10. The staff are trying to get the rest processed asap, but it seems to take ages just to do one order. They are concerned that if we don't get invoices out, we get no incoming money. I know that we have managed to maintain a reasonable level of cash flow previously, but that could change in a few more weeks if we are not careful.

A big problem has occured at one of our other sites. They were struggling anyway - people were taking too long to process anything at all. I spoke to the GM on site and he was getting really quite anxious - he did indicate that they might have to start turning away orders as they just couldn't get the work done.

This has been made worse as he has now lost all but two of his office based sales staff - they just upped and quit in the middle of the week, saying that they'd had enough. We've been asked to send staff to him to help out, and it is causing some major issues with arrangements. I did also hear that a couple of others have also indicated their intention to leave - this could get really serious.

One good thing tho' - a decision was finally made about the CRM system. After 2 years of work, it's been agreed that the SAP CRM doesn't meet our needs. The way that it runs is just too slow, and the sales people think that the interface is really nasty (it doesn't show what they need on the main screen). It also doesn't produce the information that our Sales Director said was a fundametal requirement.

We have had 4 different consultants workng on the CRM since the beginning - and basically, we have nothing to show for the time that they spent on the processes. The SAPGUI part runs OK, but they said that it had to be used through a web browser - and that was where the problems were. The speed of loading was really poor, up to a minute to move between screens at times.

Over the last 2 years, they have made all sorts of excuses - not enough memory, issues with SQL settings, AV products, the only thing they didn't blame was the phase of the moon! A while ago, they indicated that they wanted to get some more people in to work on the problems, but that they would want more money to do so. The CEO was not impressed - so he has made the decison to forget about the CRM, and the sales staff are happy with that.

The consultant company sent us some documents recently about arranging for support after the go live process. We are alreading paying for SAP support (close to $100,000 pa) and they want us to pay money to them as well. They won't actually come up with a price, but just give loads of .ppt slide with the "benefits" for having them as a support contact. From what I have seen, it could double our support costs - but there would be no reduction in our staff to compensate.

Oh well, tomorrow is another day!

Did I speak too soon?

The first couple of days this days this week went OK - pretty much like last week. But then we hit some problems.

The issue with the documents printing out extra pages is still going on, although we have one of the consultants back in to help us. He's been looking at the specifics, but hasn't found anything so far. He originally suggested that we had faulty settings on the print server, but after making us double check it all, he had to admit that there is nothing wrong there.

On Wednesday, one of the staff in sales was putting orders on the system and suddenly realised that the prices were coming out incorrectly - she only spotted this as she has been doing the job for close to 15 years and can tell if a price is close or not, which these weren't. They spent the next 2 days checking this out but so far, no-one can see where it's going wrong. All we know is that the prices loaded were correct.

We also had a problem with the production plan on Thursday - for some reason several orders were entered twice (we haven't yet found out why) and this caused the production schedule to change to deal with an unanticipated demand. The system is supposed to detect if a customer's reference number is entered twice, but it didn't flag this up as it is supposed to do.

I've been busy with changes to user roles - I'm still getting just under a dozen a day. It's starting to get tedious, but one of the consultants said that this is quite common. He indicated that one project he was on had changes being made to the user roles every day 3-4 months after their go-live. I sure hope that's not the case with us as I have lots of other work that I need to be doing. Most of this is stuff that has been delayed because of the SAP implementation and we need to be getting on with catching up.

I've also heard some complaints from senior managers. They want specific reports daily / weekly and what they are getting is not what they want. We were told by the consultants that all of this was standard and they gave us transaction codes for reports - but it turns out that some of these don't actually give us what we asked for. One in particular is a report that indicates sales invoiced, but the figures that it is producing make no sense at all. We also had a labor report that seems to indicate that efficiency levels are about a quarter of what they were a few months ago.

On the plus side, people are starting to get to grips with the program. I don't get so many people locking themselves out because they entered the wrong password. They are still slow at using it though, but that's probably because they are still finding their way around; it's going to take a few months for them to pick up some speed. We had a few complaints about speed on a couple of occasions, but we were able to identify that someone else was running a report or other process that was tying up resources - hopefully, we have now put a stop to that.

In particular, we found some workflow items that had been set-up by a consultant that were failing every time they ran. We had run into this before and with the SAP support had found out how to correct this; we were able to do the same this time.

In addition, we had some feedback from the SAP Earlywatch check. They basically indicate that our systems have more than enough processing power / memory to run the program. This contradicts what the director of the consultants tried to tell us some months back (he insisted that we didn't have enough memory). They also made some suggestion about settings which we have now implemented, and some others about security permissions which we can't just yet.

My staff and I now have a basic routine in place to check on the system performance although I think we need a bit more experience to be able to understand the results better. Unfortunately, we have nothing to judge the results against - I have no way of knowing if we are doing a good job or not, or if the results are good, bad or indifferent.

Oh well, such is life. I suppose that these are mostly the types of teething problems that everyone gets with a new product.

Week 1

Well, the first 7 days have gone by, and so far things are not looking too bad. If it carries on like this, then I for one will be quite pleased.

First thing on Monday morning, I was in very early. I'd made a point of getting as many people as possible to logon in last week to make sure that they knew their username / password and that they could actually get access to the production system. However, I suspected that there would be a few phones calls and I wasn't dissappointed! I had about 5 people had managed to forget their password and completely lock themselves out of the system - I was able to talk them thru getting back on. This was fewer than expected, so I was quite pleased.

Later, I had a few questions about access permissions - although we had spent a lot of time working these out, it was clear that there would still be issues after we had gone live. Some of the decisions proved to be a bit optimistic and it was found that we needed to give some of the lower level staff access to more than had been originally agreed. Again, I expected this, so it wasn't that big a deal for me - but it was for some of the managers, as they want everything done imediately and with SAP that is not always possible.

We've developed a reasonably satisfactory process for getting these changes authorised, but people still try to bypass the process and get very irate when I send their requests back to them with the instructions. However, they will learn eventually. I am getting more than I expected, about 8 -10 a day. Hopefully that will tail off in the next week, but I'm not convinced it will as I'm sure that they didn't check all of the variants prior to go-live.

We had one major discussion - a few weeks ago, there was a request for a change which I said was a bad move as it could cause a potential problem. That request was then reversed a few days later as our Operations guy found out for himself just how bad it could be. But then a week later, he changed his mind again - we needed this to allow the project team to push things thru. Then there was huge problem caused by one of these guys using the permissions inappropriately, so it was taken off - and then added back on again a week later. We've had an SAP Earlywatch report and they identified that this was major issue and that we should remove this permission - so I had to sit in a room and explain to the CEO why the project team had been given it in the first place!

There's one rather odd problem with the Sales documents - for some reason, they are printing an extra blank page in between the normal pages, so a 3 page page document comes out as 6 pages. We've got one of the consultants coming in ths week and it's been highlighted that they should look at this - I suspect that it's a formatting issue, but I haven't had the chance to work with some of these documents so I'm reluctant to play around with it at this stage.

The SAP production system is working well. So far, no complaints about speed. It's clear that some staff didn't get enough training as they are taking a lot longer to carry out a function than they should - again, I suspected that this would be the case. I felt that many had not had sufficient actual hands on work, but it was difficult to fit this in. I'm sure that they will pick up speed, they just need to concentrate on using the product.

One thing that I'm bothered about, is that some staff are still using the old legacy systems. They said that they were just referring to old data, but why would they need to? The same data is in the new system, and is just as available. I think it is a bit of a comfort blanket, and as such, I feel that it is probably OK to leave them alone for the present. However, I don't feel that we should let them use these older systems for too much longer as the data will quickly get out of step.

Anyway, so far, so good. There have not been the more serious problems that I dreaded, so this weekend I've had the chance to relax and take things easier. We are not out of the woods yet, but if it continues as it all did this week, then I for one will be quite relieved and very happy.

Here we go

Well, that's it. No more prevaricating, no more delays. Tomorrow is the day - SAP go live day. No further hold ups, no excuses. It doesn't matter if we are ready or not, here we go.

The last few weeks have been crazy - the amount of work, mind numbing. Everything else went out of the window apart from the SAP project. All calls from suppliers and cold callers were turned away - all meetings cancelled apart from the project team meetings. And the last couple of weekends, we have all worked our butts off, to do the best we can to make sure we are ready.

There have been a few issues with the data load - it started OK, but then one set failed and it took several hours to find the problem. It turned out that one of the consultants had made some changes in the Development system which had been transported to the Test system but not the Production system. As a result, the data load worked in one, but fouled up in the other. OK that got fixed, but we uncovered some more items like that which delayed the process each time. We started to get a bit worried - in the end, all of the static data is done, we just have a few items left to do, but they are live data items so can be loaded over the next few days. W also found that some pricing information was way out - a change had been made to some data, and no-one knew about it. A minor issue, but it held things up becase we had to find out why.

There was also an issue with some of the product information - the consultants had been asked to make a minor change to allow for a particular piece of data we need to allow us to report on in a particular way - they had done some work, but it was wrong and we only uncovered this when the production manager ran a report. It hasn't been fixed yet, but they say that they'll have a solution in the next 2 days.

There was also a bit of friction between finance and sales - the sales people had cleaned up their data, but left a load of customer information in as they wanted to be able to track marketing effectiveness. The problem was that when finance started to look at it, they found some companies that no longer existed due to M & A activity, plus some that have unfortunately closed due to the economy. I can see the problem from both sides and normally I don't think it would be that big a deal, but everyone is now getting very tense, so smaller issues get blown up out of proportion.

The consultants got a guy to do a check of the hardware and configuration to make sure it's all OK. He sent thru some suggestions for parameter changes which I did on Friday. I checked and all was fine - but first thing Saturday, I got a call to tell me the system had fallen over. I quickly restarted and it seemed all right, but thought I should go in to be safe - a good job too as it fell over again. I looked at the log and realised that it was referring to a problem with one of the parameter changes, the value they had given me was totally out. Having put it back to the way it was, everything carried on ticking over.

The project team have been doing a lot of data check and testing of processes, far more than the consultants said we needed, but not as much as I think we need. I'm not convinced that the guys have been able to test all of the different scenarios or variants - but there's no time left to do any more.

My eldest was invited to a party for a friends birthday - her father was throwing a barbecue, so after I finished yesterday, I dropped by. It was really busy and it was nice just to sit with a beer with the sun going down and not have to think about too much. However, I got talking to to guy that used to work for a company that installed SAP a few years ago. We compared stories and although it was an earlier version, and different consultants, it was a strangely familiar tale. I am glad that it's not just us - it may sound crazy, but it helps when you look at something that's wrong and you can feel that other people have been here before.

Any way, there we are. In total, we have spent just over 3 years on this project, just under 30 months of that with SAP. I haven't seen the final costs and I know that the FD is holding back a couple of invoices and he says he won't pay them to make sure that we don't get left without access to proper consultant support (although I suspect we might be better off going elsewhere). However, the project cost seems to be a little under $1.5 million for the consultants and software - we think that with all the work our people have put in, we have spent another $750,000. Has it been worth it? Only time will tell.

Going to get a good nights sleep - early start tomorrow. I'm not sure how I really feel about it, it almost seems to be unreal. I really don't know what to expect - I'm hoping it will go well, but just a bit afraid that it won't. Still it's too late to worry about it know, just got to get on and deal with it the best we can. I think that the coffee machine is going to be working overtime this week.

I'll try to post more over the next few weeks so that you can see how things turn out.

Data loading

As you might guess, we are in the process of loading data into the production system, ready for the final tests before going live. The last couple of weeks have been frantic - we are just over half way there.

There have been a few issues - we had a big problem because the consultants hadn't transported a configuration change. The data wouldn't load and we had to wait for the consultant to process the change before we could carry on. There was another open issue for which they finally came up with an answer only for us to find that we had to modify the data load file in order to deal with this new item.

There has also been an issue with one of the tests - it's producing the wrong output figures. We are not sure why as in the tests in the test system, it worked fine. We are still checking this one out.

The guys at one of our other sites has also hit a snag. They wanted an output document for inventroy control - long term, they want a barcode on the paper so that we can use a scanner. Unfortunately, the document that they specified is producing completely wrong data; again we're not sure why.

In addition, there has been a problem emailing paperwork to customers / suppliers. It worked on the test system, but for some reason not yet discovered, it doesn't work on the production system.

Everyone is putting in a lot of overtime - we are all getting very tired and starting to make stupid mistakes. I think everyone needs to take a vacation after this.

The final stretch?

A couple of weeks ago, I had some contact with a guy that has been in consulting for a long time – he was kind enough to say some goods things about my writing, so I’ve decided to return the favor. Catch his blog here: http://sapsearchlight.blogspot.com/

One thing that I did find of interest; he referred to a process of project management that is supposed to be used by the consultants – ASAP (AcceleratedSAP I believe it stands for) also known more recently as Focus ASAP / ASAP Focus depending on where you get your info. He said “Outside of IBM and Accenture, all certified partners MUST adhere to SAP’s ASAP methodology, sometimes referred to (recently) as Focus ASAP. Most of these partners add some of their secret sauce to the core SAP methodology. I am willing to bet that if you look at this firm’s proposal of services to you that they make a big deal about their methodology.”

I did actually look thru all of the paperwork from these consultants – nowhere does it a make a mention of this. I asked around our project team to see if anyone had heard it cited, and the general answer was a definite “No” – apart from one person who remembered reading a reference to “Focus ASAP” in one of the SAPpress books that we bought a ways back. So I then thought I’d ask their Project Manager – unfortunately, he’s “not available” at the moment (I don’t know why) and we are not sure when we will next see him. I approached one of the other consulting staff and asked the question – the response was along the lines of “Oh that was used about 10 years ago, but no-one uses that anymore, it’s a really horrible system”. Interesting?

-----------------

As you may imagine, things are hotting up. For many people the project is starting to get real and you can sense the panic growing. I’ve had numerous project members and departmental heads speak to me and even a couple of the board expressing their concerns. We have no formal contingency plans other than those that I and my IT staff have discussed. We are committed to go-live and nothing will stop or delay that now.

A major problem was uncovered in the last week that has created a lot of bad feeling. As I’ve indicated, we trade across borders so use multiple currencies. We suddenly found that all of the customers had a default currency set and that this overrode any attempt to set the right currency. The initial suggestion was that the data was wrong – a quick look at the files showed that wasn’t the case. So we were blamed for loading it incorrectly, but that was quickly disproven.

Eventually, it turned out that the consultants had set the appropriate configuration in the test system, but not in the production system – thank goodness we found it before we started issuing invoices. They blamed us first for not raising it as an issue – however we did, over a year ago and could prove this by referring to the list of issues. They then said that the various departments hadn’t tested the process correctly – they had, but obviously not in the production system. No matter what we showed, they still tried to maintain it wasn’t their problem. And I’m betting that after we go live, all of the problems we then find, they will say the same thing, no matter what the problem.

In the past few weeks, I’ve been doing some travelling to the other sites (partly why I’ve not been adding to the blog) and I’ve heard of some other stories of SAP implementations going on at the moment. What is really surprising is that they are being marketed as successful, and yet from what the people on the ground are saying, they are anything but.

In one case, its being run by one of the Big 3, the others are smaller operations with smaller consultancies. Not one has come in on time, not one on budget, not even close. In each case, it seems highly improbable that the organization concerned will ever see the ROI that they wanted or were promised. At the biggest one, they went live last year and they were expecting some significant reductions in staff because of efficiencies, but instead they have had to take more staff on in order to manually process items because the system isn’t working right. And in that particular case, they’ve outsourced the data center which they expected would save them nearly $1 million a year – the local paper has found that the outsourced costs have increased their bill, not decreased it.

Well enough for now – I expect that over the next few weeks, I’m going to be very busy, so I may not have the time to keep you updated on progress. However, I may try to post a few short items if I can just so that you see how things pan out. What ever happens, many thanks to you all for reading this and for the great comments - they have been really helpful and I hope that I can return the favor one day.

I've got a bad feeling....

Things at work have started to get very tense - people are now starting to realise that the go-live date is getting closer. There are still so many items that have yet to be fixed up, and despite all the effort, it seems that many of these will not resolved in time. So the decison has been made that we will go ahead whatever the cost and live with any issues.

No doubt, many of you that read this will have heard similar stories to the ones that I have heard - about projects that suffer problems. I see it almost like a slow motion train wreck, and there is nothing that I can do.

The PM from the consultants was in the other day - he was supposed to be spending time with the finance people to go thru the closing of period / month / year. When he discovered that they didn't have the right transaction code, he didn't try to resolve the problem or ask me to deal with it - he just logged on as himself and added SAP_ALL to their user accounts. For those that don't know, SAP_ALL is a profile that when it is added to a user account gives that user full administrative access to the whole system.

I found out about this a few days later when I was making yet more changes - to my horror I found that some 25 accounts had been modified to give them the same access level. I've been told that this is needed because they no longer have time to waste and need to get these processes working and that we will fix it all up about a month after go-live. Call me cynical, but I suspect that won't happen and I'd give 50 bucks that they still have the same access level at the start of next year.

I also spotted something earlier - we have yet another new consultant on site to address a couple of specific issues. I wasn't told about this until after he had been there 3 days. I queried how he had been doing any work and was told that he was using the account of one of the other consultants - in addition he has created duplicates of the accounts our project team members use, and he has been using these to test things; but as they haven't been added to specific roles, he just added SAP_ALL.

In addition, it was highlighted that another consultant has been modifying some items - he has made a mess of it and now that item doesn't work at all. They've gotten someone else in to address the issue, but it appears that it will take 3-4 weeks to fix.

There was a review meeting earlier - as we went around the table, it was identified that one of the senior managers wasn't there as he is away on holiday. A comment was made that it was unacceptable that this key person should be away at such a crucial time. It was pointed out the person concerned is on his honeymoon - it was booked up 18 months ago before the date of the go-live had been put back so many times. In addition, the person concerned has had very little training - I think I am the only one that has spent time with him at all (altho I might be wrong). Certainly, he has actually had very little input into the project.

What happened then was astonishing - I tried to point out that this was a bit unfair and one of the directors started shouting and threw his notepad across the table at me (I suppose that I should be greatful it wasn't his laptop - no doubt I would have had to repair it). The rest of the meeting was extremely uncomfortable as you might guess.

I could post more, but just don't feel that it's worth while. I suppose that when it goes belly up, then they will be demanding that I wave my magic wand and fix it all in 10 minutes. I made a point of reading Rudyard Kiplings "If" - I think that it has a lot of resonance for those of us in IT, even tho it was written some 100 years ago.

http://www.kipling.org.uk/poems_if.htm

Busy, busy, busy....

I was going to write something last week but just couldn't make the time. It's been pretty hectic this week as well.

Part of the problem is that we have to do the SAP work at the same time as our normal tasks. We have to maintain hardware, patch software, keep the AV running, check backups, hold people's hands when their printers don't work etc. In addition, we have a 5 year development plan that gets updated every 6 months or so and we try to make sure that we stay on track. Over the last couple of weeks, there have been some changes to buildings and we were moving hardware around, doing some cable patching etc. This is straight forward if it is planned out - however, some of our managers think forward planning is deciding what we will have for brunch!

The SAP data loading has started - I've made the point to people several times that now this has started, we can't really delay the go-live date. Of course if necessary we could, but it would cause a large number of problems. However, we want to concentrate their minds and make sure that they understand the importance of the process. Despite this, some still haven't gotten with the program - we loaded some price information only to be told a day later that they wanted to make some changes!

Now I did the intial tests of the loading process way back and setup the scripts to import data. One of my staff was put in charge of the data assembly and he's done an excellent job. He's also tested the script in the test client to make sure that it should all run OK. So when we started the load process in the production system, we expected it to go well - and for the most part it did. Unfortunately tho' we hit a couple of snags.

For those that don't know SAP, you have 3 systems, Development, Test and Production. In those you have "clients" - effectively instances that can be run separately. One of these is designated the master client and all changes are made in there - then a system of "transports" move the configuration changes so that all systems are the same. It turns out that several of the consultants have been making changes to the configuration, but then haven't transported it to the production client correctly. Worse, they didn't even put the changes in the master client, so any transports will over write what they've done. We've had to go thru and sort these out as we find them, which is slowing down the load process - so lots of late nights and weekend working.

On top of that there are still a large number of outstanding issues, and one of these is to do with the finances. Apparently the consultant for that area told the project manager that he was intending to deal with it after go-live. I wish I'd been there to hear the rest of the conversation!

We still haven't gotten the labeling issue totally resolved and the PM wants one of the consultants back on site to deal with this. They've complained that they wanted to deal with this a few weeks ago, but couldn't as we were dealing with other matters. They can be a real PITA at times - we have to wait what seems an eternity for them to get back to us, but when they finally do, they expect us to leap into instant action at a single word.

I'm still waiting to find out what they intend to do with the CRM - it still runs very slowly and there are numerous parts to it that don't meet our needs. The sales director has indicated that if they don't get it fixed soon, he will make the decision to toss the project. As far as he is concerned, he would rather that we use the existing product as that works. We can't link it to SAP, but that doesn't bother him.

Several of the project team have started going back over the training with their staff - I've been suggesting that they should do this for months as most staff have forgotten everything they were told. There were a couple of sessions last week - of 19 staff in training, 3 could remember their password, the others all had to be reset! However, once they went thru the basics again, they stated to recall some of the steps. Unfortunately, I don't know if we are going to be able to get everyone back in the training room. We'll just have to wait and see.

Onward and upward!

And it's all gone quiet

I haven’t posted for a couple of weeks – well, that’s mainly because not a lot has been happening. So I thought that I would go back in time a bit, and cover some of the main issues from the early days in a bit more detail.

We had a launch meeting over 2 years ago now (unbelievable how the time has flown) and it had been agreed that at that meeting, the consultants would discuss the various steps of the project, and it was expected that we would come away with a clear idea of what had to be done, who was to do what jobs and have a timetable that we would all work to. Well, that WAS the plan.

I still have my notes from that meeting – one of my first points was that the agenda that they put up on the screen was not the one agreed with the project manager. In fact, it only had 4 bullet points on it – nothing else at all. When you consider that this was for a meeting that was expected to last all day, I felt that it was a bit light on detail.

I also made a couple of other comments after about 30 and 45 minutes – “when is he going to get down to it?” You see, the meeting was being managed by the director from the consultants, and he was doing most of the talking. That first hour, he did a LOT of talking – but at the end, I had no notes as he had not covered a single item that was actually relevant to the project plan. I could see that a couple of our senior managers were getting restless and the CEO left twice to take phone calls.

We broke for coffee at 11:10 – my real notes by that time were on 5 lines although I had some real nice drawings, none of which had anything to do with the project. During the break, our CEO and the consultants’ director got together and I’m told the CEO told him to move the pace up a bit. But it has to be said that by lunch, we still hadn’t really any better idea of what the project plan was. My favorite part was when the guy turned to the CEO and suggested that we ought to have a few bottles of champagne to get the project off to a good start – of course I had to speak up and suggest that perhaps champagne might be more appropriate once the software was actually running. Funny that he and I have never got on since.

After lunch, we were back to this guy talking on and on without actually making any real positive comments that would have been of any value. Later, just before the afternoon coffee break, he allowed their project manager to say a few words. This guy did actually discuss the timetable – but took no more than 20 minutes. Essentially, he stated that there were to be 6 key milestones (there were more, but he only referred to the 6) and he gave the dates for those, one of which was the meeting we were at (10 days late already) and the last one was for the go-live date.

By the end of the day, all we really had was a very basic timetable showing the start and finish with the “blueprint” phase, acceptance of this, the data cleansing, data loading and cutover to the live system. The idea of data cleansing had been briefly covered, but in very limited detail. I had suggested that we should start this as soon as possible as I knew we had a lot to do – this was brushed aside and in fact it didn’t start for a further 4 months. I should highlight that we are now some 18 months on and today it was found that one particular set of data still hasn’t been corrected and won’t load in its current state even though it has been sent back to the relevant department 5 times.

It was a further 2 months before most of the project team even really began to know what was expected of them; none of their responsibilities had been discussed and nothing was down in writing. I know that I can be a bit anal, but for me this so important. People have to know and understand what is expected of them – most won’t make notes, so I feel that writing it down so that they can refer to it is a must.

If I were to ever be involved in something like this again, I know that my priority would be to make sure that the launch meeting set the tempo for the project – that within 30 minutes, everyone would know exactly what was expected of them as individuals and of the team as a whole. We would have clear definition of responsibilities written down, with due dates and standards expected.

I think it a shame that the experience that I have gained from this project will never be used to make another project more effective – that is partly why I’ve been writing this blog, as I hope that other people might come across it and find it of value. The skills that I have gained have come at a high price in terms of workload and frustration and it should be possible to help other people avoid this. Certainly from what I have seen, the consultants have learnt little from it and are most likely to repeat the same errors, over and again.

Oh well, back to the grindstone.

The seconds tick away…

I’ve indicated that the company has a number of sites in this country as well as several more overseas. The original plan was that we would implement the SAP program and once it was proven to be working, it would then be rolled out to the other sites. There was a slight change to this in that we bought out a company in another country and for a while, it was required that they would be part of the project; that was cancelled, so we are back to just the original sites.

It should be highlighted that each site sells different products – yes there is a commonality, but the difference is sufficient that they use diverse processes. Now in the past, this has caused an issue – it’s been difficult to get any agreement on standardization - this is why we wanted a single ERP system.

When the project was set-up, the consultants suggested that we create a project team made up of managers to represent the key areas within the business. It was decided that our director of operations would head the team and act as project manager from our side; he would liaise with the project manager supplied from the consultants.

So far, so good - however, I queried one thing; most of the project team were from the one site, and there was only one representative for the other 3 sites. The explanation for this was that as we would all use the same processes, the one person plus a single consultant was required due to a specific area which they dealt with and that the other sites didn’t. I understand the idea of this, but I felt that this was not right.

Although we need to try to get some consistency, the nature of the differences in the way that each site works is fundamental and many of the business processes are then affected by this. I was concerned that we would see the system set-up to work one way only and that the other sites would then struggle to use a system that could not handle their particular needs.

After the blueprint phase, I was even more convinced that the one person would not be able to deal with all of the issues on his own. As we moved through the next steps, it became obvious that he was struggling to keep up. About 16 / 17 months ago, it was decided that he needed help and he has had 4 more people helping him, 2 on a temporary basis and 2 permanently. Despite this, I still felt uneasy – they were still concentrating only on certain key steps and ignoring other modules on the basis that the project team would cover those items.

Now over the past 8 / 9 months, the guys and gals from the project team have worked extremely hard to train people from the other sites. During this phase, there have been a number of questions raised by the people being trained about the way that they are to work in future. Some of these questions are not really relevant – we have been able to simplify some processes and things that they used to do are no longer necessary. But unfortunately that’s not so in each case and there are a number of key items that are absolutely required that still have yet to be set-up properly.

And now it gets worse. Our project manager went to one of the other sites 2 days ago to meet with the managers from those other sites. He showed them a PowerPoint Presentation and then ran through a demo of SAP using some data that they would recognize so that they could see that it worked. He told yesterday that the presentation went well and that these managers are now really happy with it and they all feel very positive. The presentation lasted a total of less than two hours.

I queried if these managers had actually used the software themselves – the answer was no. He sort of said that they wouldn’t be the ones to use it, so it wasn’t important that they knew the software, but that they should understand what it could do.

I have to disagree – the people concerned are the site director, who needs to be able to get sales data, the production manager, who needs to know how to schedule and manage the production, the sales support manager who will have a major task in ensuring that products are produced, packed and shipped on time, and one of the senior managers that is involved in ensuring key accounts are dealt with correctly. I know that of these guys, 2 have never logged on to SAP, the others have logged on, but neither has used it in the last 6 months.

Although he has said he is confident that they are happy with things now, that is not what I am hearing from other conversations. I really am concerned that these senior managers are now completely out of the loop – and worse, because they don’t understand, when the staff have problems, these managers will not know what to do or how to find a resolution.

At this stage, we are committed to a go-live date – the CEO has said that we will not delay any further under any circumstances. If it is not ready, it is not ready but we will just have to make do with it in the condition that it is in. (I actually believe that there is now some political pressure from somewhere being applied to force the project forward whatever the cost.) Whatever the truth, I do see that sometimes it is necessary to force the situation, but I am troubled that there are too many pitfalls and things may get ugly. Time to clean up that old fall-out shelter!

Thru a glass darkly

It occurred to me that when writing these posts, I've fallen in the same trap as a lot of people; I make assumptions that the readers will know certain things about SAP that I don't need to elaborate on. So I thought that I would clarify a few issues.

SAP is an acronym - Systems, Applications & Processing. The title was originally in German, but it works the same in English. It is both the name of the company (SAP GmbH or SAP Inc etc.) and it is also the name of the software product that the company produces. Normally, it is fairly clear which of the two is being referred to, but just occasionally, it can be a bit confusing.

The company is a global player - they operate in almost every market area of the world. Previously, due to the size, complexity and cost of implementing their product, they concentrated on selling to the larger organisations. However, some years ago, they realised that there were a lot more smaller businesses out there than larger ones and so changed their focus somewhat - they now try to sell to all sizes and type of organisations.

The software is a massive product, with many modules covering different areas within a business. It is intended to be a true Enterprise Resource Planning (ERP) product; that is, it allows data from all areas of a business to be handled and processed in order to allow a seamless transition of data and therefore much greater accuracy of processing and transparency of data for analysis. This has previously been of real importance to the larger operations, but is becoming equally important to the smaller ones that wish to compete on a equal footing.

So far, so good. However, now it starts to get a bit more complex. You see, SAP (Inc) don't actually do most of the selling of SAP (product) . Yes they have a massive marketing budget and teams of specialists that help in the selling process, but the majority of the selling is really done by other companies; and these are the ones that also (generally) do the implementation of the product. These can be smaller companies operating in a single country or larger one that operate across a global region - it could also be one the big boys such as the major consultancies. This actually also includes people such HP & IBM who are surprisingly active in this area.

Now there are people out there that will tell you that SAP (the product) can operate "out of the box". I know what they mean, but in fact that is a very simplistic statement and in reality is simply not true in the way that say Microsoft Office would work out of the box. From my experience, I can say with utter certainty that the software installation is not a just case of putting a CD / DVD in the drive (they actually supply a large box full of over 100 discs!). In order to carry out even the most basic of installations successfully, you need to know certain things about the product and your business before you start. Even then, there are a number of factors that could prevent an installation taking place at all.

In order to use the product effectively, it also needs to be set-up - SAP (the company) refer to this as "best practice". What they actually mean is that over the years, they have developed a series of modifications and changes to meet the needs of the larger businesses that they have installed the product in. They have a wide range of these based upon sector and process type; they can select which of these to install on a "mix and match" method. This requires a good knowledge of the product AND of the business - and I would suggest that this one area is responsible for many of the problems that occur during implementation.

Now it gets even murkier. In some of the consultancies, the people that work for them are employees, but many are only on short term contracts - say 18 months to 2 years. These consultancies will hire people for the specific project based upon need and skill. Unfortunately, many of the people hired are of varying quality - and in some cases, they end up working in a module area that they are not particularly skilled in. I've also seen that the consultancy can hire in a person who is in turn a private contractor, and these generally get paid by the day or week. I also understand that in some cases, the larger consultancy contracts out part of the work to a smaller firm. And of course, the smaller firm can then hire in the actual people, who might even be employed by yet another business.

You see, some of these consultancies specialize in specific areas - HR, Accounts, or groups of modules such as those that make up the manufacturing process. The concept is fairly sound - you hire in the expertise in the area that you have the need. This allows you to act as a consultant even if you don't have the required specialization.

But with all of these different people, using different employment methods and structures of reporting, you find communication problems even in the relatively simple projects. From experience, many of the individuals like to work in particular way and use specific methods or practices - in some cases, these clearly don't meet the specific requirements, and I've seen that they often go against the defined goals.

Now, for a lot of people, this will seem a strange way of working - but you have to understand that this is the SAP business model. They are NOT in the business of selling software - they actually want to sell the knowledge and experience of the consultants, either as business consultants, project management, training, education services - basically all of the additional items that are perhaps a bit harder to quantify.

Don't get me wrong, when the software is set-up properly, it seems to work well (if perhaps a bit long winded). However, I would suggest that the way that the product is sold and then managed is almost designed to lead to issues that will then require the hiring of additional outside expertise. The problem then of course is that often people get locked in to the product - no-one (especially in senior management) wants to admit that they might have made a mistake, so they act like the gambler who keeps on playing in the hope that the one big win will cover all his losses.

Our project is one of the less expensive ones - we've spent nearly $2 million so far after 2 years and based upon some of the comments and observations, I expect that we will see an annual expenditure of another $100k to $150k for the next 5 years in addition to any contracted support cost (which was hiked from the beginning of the year). I still have a copy of the original documents - they stated categorically that the total cost over the first 5 years would be $1.06 million and this covered all installation costs, support and consultancy.

In reality, we will probably have spent around 3 times as much by the time that we are finished. I did some basic numbers and to get the return that they suggested, we would have to continue to use the product for around 18-20 years, and that doesn't allow for any subsequent changes or projects. I have to be honest, I cannot see that we will ever actually get a return on our investment - it's just going be a big old money pit.

It's a lovely day...

....for sitting in the sun, maybe move into the shade a bit later, and share some bread & wine with the family and just generally enjoy a very pleasant weekend. Maybe a barbecue on Sunday?

However, part of the time I've spent doing some SAP related work. I discovered a minor issue on one of the servers - not enough to cause a big problem, but I want to try to eliminate any of these little errors before we go live - I think that we'll have enough to do once we achieve liftoff.

Now the actual SAP global support people are pretty good for the most part - the method of reporting errors is a bit of a pain at times, but once you get someone on the case, they've shown that they can get things done. So far, they've managed to fix almost all of the issues that I've raised - the biggest problem I see is trying to identify the correct function area for the problem to be listed under and they can be a bit sharp with you if you enter the wrong data.

On this occasion, it's actually been passed through 3 different support people - as each one has looked at it, they've suggested that the problem is caused by something different. The last one has asked me to carry out yet another set of upgrades. As has been indicated, my staff and I have had very little real training and the instructions that we received were so vague that they proved to be of little value. I have however been given the email address of a guy in Germany that is prepared to help - he writes / speaks pretty good English (a good job as Ich verstehe nur ein bission Deutche!)

The information that he provided for how to download and carry out the upgrades was not perfect - however, after reading and checking the information carefully, I have been able to use it to work out the correct procedure. I've also been able to create my own training document on how to do this specific task so any of my guys could do it if I'm not available. For me it's important that we don't rely on any single person's knowledge - it's too risky and I've seen companies that suffered because they were stuck when a key employee left them.

So on Friday, I managed to get the test system upgraded to the latest relase of the kernel. I actually am quite surprised that it went so well - prevously, the work was done by one of the basis people from the consultants and we had all sorts of problems both during and after the work was done. On this occasion, the update worked so well, that I just couldn't believe it. I literally spent about an hour trying to find a problem before I was prepared to think that maybe it had completed successfully. I then carried out the upgrade on the other systems after everyone had finished for the day and by late on Friday, it was all done. Subsequently, I carried out a system transport of the use role profiles. Again, it all went quite well, so we can start Monday with a freshly patched system.

Now this may not seem like a big deal - for many people, patching and updates are a pretty simple process and one that most users don't see too much of. Within the company, we run an automated patching solution for all of the PCs and servers that takes care of the OS, the productivity suite, and most of the other software products we use and that has saved time and money over the years. It also ensures that as far as possible, we are protected against malware - an ounce of prevention is better than a pound of cure.

But for me, the ability to be able to this work without having to rely on the outside consultants is a major step forward. I don't think that I can understate just how much better that makes me feel - stupid I know, but having been thru such a long process with so many problems and so many failed areas, this is a real achievement even if it is such a small success. Now this doesn't mean that finally all is well - I wish! But at this point in time, I'm prepared to take anything positive that I can find and use.

So now, the work is done - time to relax and feel good about having finally achieved something positive. Now if we could just get the rest of it working the same way.....

-------------------------

BTW, I see that I have a couple more followers - welcome to you all. I hope that you enjoy reading my comments, and find them of some value. I will keep on writing as things happen; I think that we are now pretty committed to going live later this year (but more on that next time) and I will continue to detail what happens after the big event. So keep watching...

Another 2 cents worth..

Uhhmmm...

We had another meeting yesterday - the director from the consultants arrived with a new guy in tow. Introduced as our new CRM consultant - however, we did some investigation and it turns out that he last did CRM some 5 years ago with version 4 of the product. Since then, he has acted as PM for most of the project he worked. So he seems like a really sensible choice (not).

During the meeting, the director from the consultants said that the new guy would be spending about 10-12 days on site to get an idea of what our requirements were. Excuse me.... what have they been doing for the last 2 years? Un - believable.

Now we were promised at the previous meeting I referred to, they would carry out a demonstration using a customer's system to show that the product works. (That was actually supposed to have been done about a week ago.) However, that still hasn't happened yet - and in addition, they said that they are now setting up a test system for us to see, and they want to make sure that it is OK before they do the demo so it will be another couple of weeks. Color me crazy, but I would have expected that to happen before we bought the damn product, not two years into the project and certainly not a couple of months before we are due to go live.

I have to say that I think the director from the consultants is so full of BS that he could fertilize the Sahara. If he told me that the sun rose in the East, I would want to get up at midnight to check it out for myself, before I would actually believe him. The funniest bit was his insistence that we are on target - we are now actually well over a year behind, but that doesn't seem to be an issue for him. He also tried to say that it's within budget - but not according to their documents, which I still have a copy of. We are in fact over spent by 3 times the original figure they quoted for a 5 year write off - and we had identified at the time that their figure were incorrect as they didn't include several key expenditures.

When I left the meeting, I just wanted to explode.....

The same consultants are also working on another project and occasionally, they can't get to us because they are on site with the other company. They started this project way before ours - not sure just how much longer it's been running. This other company went live back in the new year - I tried to contact them, but their IT manager wouldn't discuss the project with me (they had been told not to I think from the sound of it). Since then, one of our sales staff spoke to one of theirs - and they got the woman's number for me to call.

She's fairly new in the job as it appears they have lost quite a few sales staff because of the project - and she made it clear that she doesn't want to stay long either. She made some comments that no-one really seems to know how the system is supposed to work and they get all sorts of problems during the day which take ages to resolve (she referred to one problem that took 3 days to fix and they lost a huge sale because of this). Their sales people are really fed up with all the wasted time - and it also appears that they lost a couple of IT staff as well which has made the problem much worse.

I've also had a contact with another company (different consultants) - they have only implemented part of the SAP system so far - sales, purchasing and accounts. One of our people went to see them and talked about how they were getting on - they seemed to be doing OK, so it shows that it can happen. But they have also indicated that they are still getting issues - it appears that they had a problem with getting the invoicing correct. I'm not sure exactly what their problem was, but it sounds as if it was applying the wrong pricing. This is something that can happen to any company and I would suggest that most systems have done that at one stage or another, so it would be unfair to highlight this as if it was just SAP.

But even they said that the process had been painful and they were not entirely happy that they were getting all of the benefits that they expected. Questions have been raised about the cost of the project and it seems unlikely they will realise any actual savings for their investment.

Someone made a comment that bad news travels faster than good news - true and I accept that many big project fail somewhere along the line and I'm sure that there are some SAP projects that are models of success. But for me this has been by far the worst project I have ever been involved in by a long way - I've worked with some really astute business people in the past, and I know that they would have demanded heads to roll for a failure a bad as ours has been. I really can see the ax swinging at some stage - and I'm concerned that my guys and I will suffer as a result.

Oh well, tomorrow is another day...

Another day, another dollar

We had another meeting today, with just one question – "why are we still no closer to go live than we were before the New Year?" I put forward my list, the same one that I have kept since around last June; of the items listed, only one has actually been resolved. Their project manager spent about 15-20 minutes going thru the points, but at the end the only real answer for all the issues was “it hasn’t been done”. For what it’s worth, my list was actually one of the smallest – there are still over 300 items on the open issues register.

Now I will accept that my list is the only one that hasn’t actually been growing in the last 12 months – all the points I raised have been around since then at least. Of my colleagues, most have discovered more issues as we progress through the various parts of testing and correcting. However, about half of the items have been on the list for at least a year – we know because we recorded the date entered against the item. More than a few had been marked as fixed, only for us to find that they were not, or for other changes to have broken them again.

At the end of the meeting, we had actually achieved very little. Once again, lots of promises, but nothing to back up what they say. People were very quiet coming out of the meeting; in the past, they had grouped together to bitch about the lack of progress but this time, almost everyone just went off back to their departments without a word.

The FD was in one of his moods after the meeting. He wanted to talk to me because a decision has been made that wasn’t discussed. As I’ve indicated we have sites overseas – one of these was supposed to be going live shortly after us. To make that happen, we had planned a lot of changes to provide the infrastructure, most of which are now in place. It appears that this has all been pretty much wasted – the directors have decided that we will continue with the SAP project, but it seems unlikely that we are going to roll it out across the group for at least 3-4 years, and possibly not even then. (But this isn’t generally known just yet)

He showed me a document that detailed the consultant fees for that overseas site (not the same group that we work with, a different company) – just under $180,000. Apparently, the work they did was of no value as it didn’t match the site requirements, our group requirements or meet the needs of our consultants. It appears that we have no possibility of getting our money back – the company that these consultants worked for is in the local equivalent of chapter 11 and it seems probable that they will simply cease to exist.

Now I know that there are people that will say, “$180k is not a great deal for an SAP implementation” – true and I know of a company that spent $30 million, but we couldn’t afford to spend even a small part of that kind of money. To put it in context, our total group profits for the last 10 years was just over $30 million. We have a great product and excellent service - we retain a lot of customers for this reason. But we cannot justify spending that much money unless we get seriously improved efficiencies and I doubt we will see a return that will be even close.

What makes me really mad is that I have a small project I’m working on; it won’t get us any extra sales as such, but we can see the potential for better analysis to make us more labor efficient in one area and that might get a few more sales. The cost of the product? - $2495 plus tax, and this includes a half days training. However, I am having to again justify spending the money even tho’ I put it on my budget before the start of the financial year.

What’s really getting people pissed is the director from the consultants – he turns up having flown to the local airport, takes part in a 2 hour meeting then back to the airport for another flight. He promises much, but so far has delivered zip – everyone calls him the seagull, because he flies in, makes a loud noise, craps on everyone then flies off!

Better sign off – I’m waiting for a response from Global support.

The sun is shining...

It has been a particularly nice weekend - I've not had to do any SAP work (although I've been checking up on some other IT issues). We went out yesterday for a walk in the country, and it was extremely relaxing. Because of that, I'm in a slightly better mood and decided to write about some parts of the project that have been a bit more positive.

For one thing, we are getting data cleaned up. About 4 years ago, I raised the issue of data integrity and consistency and it has been a topic of conversation many times since. Although people have tried to do this at various stages, we have a large amount of really poor data in the various systems. For example, in the legacy systems we use, the customer / supplier addresses don't match, and not just the wrong zip codes - some of it is far worse. Few of the systems have consistent contact information and many of the details are out of date (people no longer working at the particular company for example). I hate to say it, but there is a similar problem with product information, manufacturing data, accounting information etc as well.

The SAP project has been a good opportunity to address this; and a considerable amount of work has been put in to tidy up the data before migrating it over into the new system. People were tasked with this and a lot has been done - although I think we could have done a better job of explaining what they needed to do as several of them didn't get it right, and we have had to send the data back to them for checking not just once or twice, but many times. A case in point was one of the sales staff dealing with pricing for export - after 3 months, he still hadn't got the right prices against the right products. In the past he had been allowed to get away with asking for changes to be made by the IT staff when errors were found, but not on this occasion.

As a result of all this work, we now have a much better idea of where some of our pricing is wrong, and we can adjust this to ensure that in future it is much better - we know that we have lost potential orders in the past as our pricing was too high, and now I think we can be a bit more competitive.

For many years now, the production team have worked on getting a leaner approach to the job, and they always have a project underway to improve processes. I know that they have used the SAP project to take a good look at some of the specific processes they have not previously covered to see if there is a better way of working. The project team in particular have benefited from this navel gazing - I think that they will need to look again at some of their processes in about a years time though, as I'm not sure that they have achieved what they need to. However, having taken a long hard look at what they do for this project, I hope that it will enable them to do it again.

Although the accounting side is still not working as it should, it has highlighted a big issue relating to reporting - this was not something that we did particularly well in the past. If we can get all of the various reports working correctly, then I expect we have a much better understanding of where our costs are coming from. At a previous company I worked for, they could accurately show what each sale cost, to indicate profit or loss - my current employer only knows that once all of the finances for the year have been finalised.

There is another small project that I am working on for our telesales people and it has highlighted some of the areas where it is clear we could do much better. We had hoped that the CRM would build upon this so that we could streamline the process. That has most definitely not worked, but the principle has been shown to be sound.

These various things have confirmed something - that most people in the business don't actually know what it is they want. Because of this, it is difficult for them to articulate their needs in a way that makes sense in IT terms. To be fair to the consultants, this has been a real problem and made worse as most of them haven't had experience in our paricular sector. However, having to go through these various steps, painful as they have been, has actually been of some benefit. Although they still have a way to go, most of the managers now do appreciate some of the issues relating to poor data, incomplete requirements definition etc. And that can't be a bad thing!

Might even head to the beach this afternoon - it looks as if it will be a great day for swimming.

Status Report

The go live is delayed again - it seems appropriate to put together an overview of where we are.

From the previous post, you'll know that the CRM just isn't working - the consultants are squirming like worms on a hot plate, but whatever they say, it is just no good in it's current format. We've tried our best, but at some point we will have to move on.

By department, Marketing probably have the least to worry about - no CRM, and nothing else has been arranged. They will almost certainly not be using SAP at all.

The Sales staff are reasonably happy with their processes in the ERP and can make quotes, place orders etc. fairly well. There is a problem in that it still doesn't price things up correctly; some of the tests have come out with really bad results, up to $100 out on a single item. This is being looked at, but it has been an issue for about 6-8 months now. There were problems with the output documents such as invoices, confirmations etc. but most of these have been rectified. The remaining issues are a nuisance, but wouldn't stop us going live - although we might look a bit stupid with some of the errors.

Purchasing is also working well for the most part - their only problems are back to back ordering and batching of purchase items which have been an issue almost from the beginning. There have also been some concerns over the releasing of purchase orders; we have found occasional glitches. However, these have not been too serious.

Within Manufacturing, most of the processes are also OK, but there has been an issue with goods movement - it still isn't right. About a third of them are having to be corrected by the senior production manager and he has been given much higher access than would normally be the case just so that he can correct these problems. They do have an issue in that some of the workshop documentation they need is still not yet working - and this could cause serious delays in the output as the staff have to wait for production managers to resolve quite simple inquiries.

The Inventory is a big problem - so far, we've not been able to get a valid figure on anything, and some of the errors are so riduculous. Literally thousands of items appear on stock figures instead of 1.

The Project Team are badly affected by this and it has proven to be the biggest failure so far. Transfer of goods between sites generates all sorts of strange anomalies. On top of that, only about 10% of the standard project tests came out even close to being correct - several have generated errors in the 10s of thousands of dollars and in one case almost half a million dollars.

Logistics have managed to work around the few issues that they had - there is a problem with the stock racking, but they can live with it. They have been waiting for their output documentation for months - they finally got it sorted about a month ago, and we've also now managed to get the address labels printing this week. However, there is still an issue with some of the output - addresses don't quite appear correctly. This is not too serious, but we would like to get it right if possible.

The finance though is a mess - the chart of accounts just didn't match what we needed. The consultant that was supposed to be their specialist didn't seem to understand some of the basic requirements. At this stage, we can just about invoice, but there are major discrepancies and no sign of how these can be resolved yet. They also don't have access to a lot of the reports that we were told they would be able to generate. It is still being worked on, but the Accounts staff are not happy at all.

The QA / QC people are also not satisfied - the defect notification aspect appears to be non operational. They have tried several times, but so far the consultants appear to have no one that has even the most basic understanding of this aspect.

There were so many other reports missing, that almost every department was concerned. We have had a specialist on site and he has fixed several of these, but not all of them. This of course is a major reason for buying an ERP system - without it, we will struggle to use it for more than a few months.

Finally, we are supposed to have a Document Management system, and yes it is installed. However, there are serious questions still unanswered about how it will work - it was supposed to automate several processes, but instead they have become more labor intensive. We were given to understand over a year ago that they had a way of scanning incoming paper based files for auto assigment - this was completely incorrect, the consultants just referred us to another company that wanted to sell us new xerox machines at twice the price we paid for the ones we have, and still use a manual system for assignment.

So there we are - the go live date has been pushed back again and is now beginning of 4th quarter 2009. When we first met with the consultants, they promised us a go live of January 2008 - I said at the time that was completely impossible, but they were so confident. The strange thing is that their director insists that they made no such promise despite us showing him the actual letters that he wrote.

Oh well, tomorrow is another day.

CRM - Sour CReaM

It's been a while - time to try to catch up....

When we first looked at SAP, the VP then in charge of sales wanted the CRM package. He completely bought into the promises - linking into ERP, all the pipeline reporting etc. We had a number of different systems used all over the group and it made sense to buy a single package that every one could use.

When the consultants did the first presentation of the CRM, I was not impressed - it just didn't work. The guy doing the demo showed the pages, but no actual data or processing. We had a second demo a couple of months later, and they didn't show the product at all.

After the project started, it was several months before they had gotten the system installed. I setup user accounts as instructed and the client software. It was only about 2 months after this that we were told we would be using a browser based version - and I was given no information on setting that up.

The "CRM specialist" provided for us finally gave a URL for the sales people to use - but it didn't actually work. I'd bought some books and one of those had some info on setting up the BSP application in the user favorites - without that, we'd never have got started. After several hours of working to try to identify why it wouldn't work thru the browser, I worked out it was a security issue - there were some settings in IE6 that needed changing (even more in IE7). Once these changed, it got us started for real.

We then ran into some more issues. It actually took the "CRM specialist" a YEAR to realise that there was a specific user role that had to be used by all users. Prior to that, I had to give everyone the SAP_ALL profile just to allow them to work in the CRM. He also failed to provide any real information on setting up the business partner relationships, which I was told were essential to getting it working; I did find some info in one of the books, but to this day, I don't know if what I did was right.

There is a component for "organizational model" - this sets out the relationships between staff so that (I think) it allows the workflow to process jobs giving the relevant alerts etc. He created this model and I've not seen a bigger pile of crap in my life. The names of staff were mixed in with the names of the consultants, plus what I think were made up names. The actual job functions were all wrong, the site details were wrong, people in the wrong user groups, wrong sites, wrong job functions - well you get the picture.

I did work out how to change this and managed to correct it - then was told that this cannot be transported between systems so has to be manually changed in each client. Having made the changes (taking several days), I was not happy when I found out that he had gone in and modified it further, making a mess of it again.

So far, it would fair to say that this was just teething troubles - but it was far worse than that. The sales staff had real problems using the CRM software - it was slow, painfully slow. It would take literally a minute to open the main page. Then to go between sections could take 20-30 seconds at a time. It was flaky as hell - the system would crash regularly even when only one person at a time was working in it. Four sales staff spent an afternoon doing some testing - in one hour they crashed it a total of 19 times. Even when it didn't crash, they found it impossible to enter data at the speed that is required when someone is on the phone giving you an order.

We moved onto the test system from the development system - and the sales manager found out that the consultant had copied over all of the crap data from the dev system. (Subsequently found that this had gone to the production system as well). It was then discovered that the CRM didn't link to the correct ERP system - this was only finally corrected 6 weeks ago - and yes that was because I had found out where it was going wrong and corrected it.

They had also promised that the CRM would link into Exchange /Outlook, but when this was tested, an error message was produced that we needed to install a CDO from Microsoft. Having checked on this, I discovered that this is a bullshit message - it's only required for earlier versions. In fact it is a security issue and can be easily corrected. However, the senior consultant kept insisteing during meetings that we were being awkward because we wouldn't install this CDO despite my showing them that trying to install the software produces a message saying it is not needed.

There were numeous messages back and forth - but they took their sweet time to answer anything, sometimes weeks and on one occasion a month. It seemed impossible to get anyone that actually knew anything about the product. They sent a guy over from Germany, who spent 2 days on site, and I'll be damned if I know what he actually achieved.

So there we are - the CRM is pretty much unusable for anything. The sales staff really don't want to waste any more time on it and I don't blame them. At this stage, they would be far better off using an Excel spreadsheet for what they want. We've had about 4 meetings between the director from the consultants and our CEO in the last 3 weeks. The last one was really nasty - our CEO has asked for our money back and has been told he won't get a red cent. That's about $80,000 down the pipes and he is pissed as hell. We've actually held off on making any payments to them in the last 4 months - and they have no chance of getting any more money from us.

Bottom line - the CRM is piss poor and we are already looking at finding another product.

User Roles

When the project started, we looked at the amount of access that users would need to get their specific jobs done. I’m well aware that if a user has too much access, then there is the potential for them (possibly quite un-intentionally) to cause a lot of damage, and made the point of explaining this in some detail. It was agreed by everyone that we needed to be really careful about what permissions were set-up.

Now having dissed the consultants on a regular basis, I am prepared to say that in this area, they did actually make some sensible suggestions. I’ve not worked with SAP before (although I’d heard about it from others) and I was prepared to be guided by their advice on this.

They suggested that I create a series of “Job Roles” which would be the basis of the permissions. Originally these were based more on their modules within SAP, but after a few tests, I felt that it was better to manage it more around our departmental structure. There is also a hierarchy of user, supervisor & manager as required. This was agreed in a meeting with the project team, way back about 2 months after we started the project. (This also actually fits well with the way that we’ve structured our AD)

The guy that showed me the process was quite helpful – he made the time to cover the main points, and then left me to get on with setting it up. He did say that it normally takes 1-2 weeks to set-up the initial user accounts and roles – in fact, it was completed within 24 hours. OK, the initial roles only had a limited number of transactions within them as that was all we had been advised of – we knew this would change as we learned more about the product.

After the blueprint phase was completed, the various consultants then started to talk in more details with the various members of the project team. From these discussions, I started to get a number of requests for additional changes. These were added at the earliest opportunity to allow people to check things out. I then found that some of what had been explained to me was slightly incorrect; so had to spend a couple of weeks going back through some of the roles to correct errors in the authorisation objects as a result. But again, that’s all right by me as I expected some of these problems.

This went on for some time; as the project team worked with the consultants, they would find out about yet more transactions that they not previously heard of – I was regularly getting 10 - 15 requests for these to be added per day (sometimes just one t-code or authorisation object, occasionally dozens). Although happy to add these, I felt that we needed to be more structured in our approach; I was concerned that I was being asked to add specific items without the need for a user to have these verified, and that this would give access that actually wasn’t needed.

I had spent some time trying to explain this to the internal project team and although they said they understood, it was clear that this was not the case. It has taken a while to get to the stage where they now really do appreciate the potential problems. In the end that I was covering this almost on a one to one basis with the guys and gals – it was the only way to make sure that they knew and I didn’t want them to feel that I was putting any of them down in front of their colleagues.

The biggest challenge for me was that the different sites in each country work in different ways – different products, different processes. Although it would require a lot more admin, to get the most value, it was thought we needed a different set of roles for each site. Now SAP standard practice is that all user roles are prefixed with a Z – they don’t use this, so if you do a search using Z*, it only produces your own roles. Sounds good – but we actually have a lot of roles, and duplicating them by country made it messy. So I suggested that we could use other letters for each country – say Y for Canada, X for Mexico etc. This has worked really well; it’s made life easier and we should get what we want from it all.

It was then suggested that we could use a similar practice for other areas – for example reports, structure etc. This was seen as a really good idea – unfortunately they didn’t stick to it. We found that we had to go back over quite a few items to correct them as the consultants hadn’t used the same structure throughout – it caused some confusion to be sure. Our PM has gone through a lot of these and straightened them out – it’s taken a while, but he is getting there.

What does worry me a bit is that I’m still getting request for changes, even after all this time. I think that part of this is due to the number of changes in consultants. I may be wrong, but they seem to each favor different methods, different transactions etc. As a result, with each change of consultant, yet more t-codes are requested. I did a quick check on the number of these – I now have in excess of 3,000 copies of requests for changes stored in a folder that I created specifically to keep track of the requests.

It’s also the case that as we do more testing, we are finding more requirements for changes. According to their original project plan, all of the changes should have been finished about 3 months before we went live – certainly before the user training began. However, as we carry out more of the training, we are finding more and more issues where the user actually needs more access than the role was given during the sessions with the consultants. I suspect that I am still going to be getting these requests for months to come.

The main thing that concerns me is the amount of time involved in all of this. During the original pitch, I specifically asked them to highlight the amount of work involved; this was to try to see if we would need additional resources. They assured our directors, that the maximum amount of work involved in administration would be 2-3 hours a week – in fact, there are 2 of us working on this and from records we have kept, we are averaging 3-4 hours a day each. This is not just user account or role admin, but server and system maintenance as well.

Note that this has to be done on top of our normal jobs – because we were told that no additional resources were needed, none have been provided. In reality, based upon what we have seen so far, we need at least one other person. I also believe that as the other sites come on line, and we need to keep system running for more of the day, we might need another person and start a shift working pattern – something we have previously avoided.

There we are - just over 2 months to go – however, there may be another delay. We will probably know more about this in about 2 weeks time.

The days and months go by..

I hadn't realised just how long ago it was that I last posted - almost a month.

A few weeks ago, it was my birthday - the kids put together a special treat for me and we planned a meal at a local restaurant. Of course, I was late home, just time to hit the shower and change, then off out with wet hair. I have to be honest, I didn't feel like celebrating, and we cut the evening a bit short. When we got home, I sat down and fell asleep. When I woke up the next morning, I was still there; and my wife had got a blanket and was there with me. I can't get over how lucky I am to be married to her - either I did something really good in a previous life, or this is a case of "pay it forward".

My wife felt that I needed to see the family physician again. He confirmed that my blood pressure is still climbing, although the cholesterol is OK (just). He wants me to take a vacation and I can't disagree with him; we are looking at taking a break later in the year. I don't care where it is - just somewhere that I can relax and forget about all this crap.

Anyway, back to work - I managed to get all of the support package updates loaded. There were a few issues afterwards, but I also managed to get those sorted as well. As I said previously, the only training I've had has been the one day early last year, and that wasn't that good. I feel that despite this, I'm actually making some ground up, although it is still very difficult. As one of the commentators said "SAP = Slow And Painful".

The project team have been working on doing end user training. As a result, there isn't a day that goes by when I don't get requests for changes to user role permissions. Most of these, I can do fairly quickly and I am at the point where I can quote most of the transport numbers for the roles - I've been doing this far too long! This has been happening for about a month now; and the general feeling is that we are not going to hit the go-live date by a long margin. This has caused considerable upset - the CEO is not happy to move it yet again. However, I don't see that he has a choice; we are still waiting for so much work to be completed. Almost every week, we find that something that was fixed is broken again. There are still issues that have been highlighted previously (one going back over a year) that still have yet to be fixed.

None of the consultants have been on site over the last month, but they are still working on stuff. I had a really nasty call from their project leader demanding to know why he couldn't get access to our system - the reason was that someone from SAP Active Support was carrying out some work to correct an error in a database table made by one of his colleagues. That has caused yet another set of emails to fly back and forth as they deny that it is anything to do with them.

Our Sales Director has also been in contact with their people. We bought their CRM system on the basis that it would be useful to have it as it links in to the ERP side and this would save a lot of issues. Unfortunately, it has been a complete disaster - the software is slow, buggy, unstable and it's been reported many times that it isn't linking correctly although the CRM consultant said that it was all set-up as it should be. I finally managed to get directions to correct the link, but it still doesn't seem to do what they promised it would. It's so slow that the sales staff would simply not be able to use when talking to customers.

The CRM runs in Internet Explorer and it crashes when more than 3 or 4 people are working on it. There have also been some issues with incorrect data being returned from some of the searches. It links to Exchange for the email, but there seems to be an issue with saving the emails and we have had some corruption of data. The reporting seems to be skewed; we have had some odd outputs, and although they said that they had the output text sorted, it's wrong again. The worst was a printout that should have been 1 page showing a customers basic sales data for the previous month - it came out as over 100 pages, most of it complete garbage.

Our CEO took this up with a director from the consultants - his response was to say that it works perfectly, that hundreds of customers use it and that it must be down to the stupidity of our staff. I'm not sure where we are going to go from here - the CEO has pretty much given them an ultimatum, it works as we need by the end of May, or he wants his money back. In the meantime, the sales people have another CRM system that they think will do the job; it is about a quarter the price of the SAP CRM.

So where we go from here, I really don't know. My guys and I are working long hours, weekends and putting all the effort in that we can; but we are getting tired, and there are some minor health issues which are starting to appear which never bothered us previously. The project manager and project team are all despondant; they are trying their best, but it just doesn't seem to be good enough. The CEO recognises this, but he is in a difficult position politically - if we delay further, it will reflect badly on him. He's not stupid though - he sees the problems and can see how bad things could be if we go before we are ready.

Well enough for now - I have to prepare a report for the next board meeting.