What a twit

What a twit: "

Amazing. Twitter has screwed up and legit users have lost followers and have been barred due to its anti-spamming house cleaning conducted last week. People can't get through to the support line and email messages are not being answered. What do you expect from a freebie service?

This is the difference between real world companies,with paid-for services, who can be held accountable and are required to provide a decent level of service, and cheap, internet-only businesses, that don't seem to give a damn. It's about time people woke up to the fact that free stuff on the internet can be pretty naff. It's crazy to think anyone would try to build a business on top of something like Twitter.

"

Complexity Science - Where it fits in

Wikipedia has a nice network diagram illustrating where "complexity science" fits in with related computation fields.

Brian Castellani provides a good survey of methods used here.

Current Research in Managing Complexity

Complex Systems Research Lab (CSRL)

at M.I.T. is researching into ways to better manage complexity in the engineering domain - focussed on aerospace, but with applicability for other areas. Here a list of research activities as of October 2009.

1. Design for safety;
2. Model-based system engineering;
3. Reusable, component-based system architectures;
4. Interactive visualization;
5. Human-centered system design;
6. System diagnosis and fault tolerance;
7. System sustainability;
8. System engineering of software-intensive systems;
9. Comprehensive Risk Management in Complex Engineered Systems

All these are the higher level research areas in software / systems engineering. CSRL are also looking at the organizational, political, and cultural aspects of system construction and operation. That, for my money, is where the "pedal hits the metal".

"Organizing for a Complex World" is a A recent good paper on this, giving a number of new insights including emphasizing flexibility and resilience in process adoption.

Also, helping us understand complex systems that already exist is the Centre for Complex Systems Research. and, in Australia, "Complex Open Systems (COS) Research Network" .

Tools for Visualising Complexity

Individuals have their own best ways to perceive information. Some of us prefer to "see" things. "Draw me a picture" you'll hear. Complexity can be somewhat reduced by having a diagram of the complex network we are studying.

Manuel Lima appreciates this point and has created visual complexity web-site visualizing complex networks for a number of domains. The index is a visual one. Nice. It's also a useful index of domains and entities that have visualizations of their networks.

Managing Complexity - Economic Rationalism

Well, the economy is a pretty complicated thing! Plenty of folk want to manage it, either to their own benefit, but, usually to everybody's benefit. And a lot of us believe that it aught to be possible to manage the economy.

Physics, Engineering, and many other disciplines, manage complexity, by modeling the thing to be managed, and then study the model, and study the disparities between the model and actuality, gaining insights, and then refining the model. Ad infinitum. This is at the heart of statistical process control. Simple idea, but not always obvious.

Economists model in all sorts of ways. Some very mathematical, some not. They come up with reasons as to why things are the way they are, and recommend approaches based on that. Sometimes they are right and sometimes not.

Thing is, when the economies of the world go wrong, a lot of folk encounter misery. So it would be really good if we could get things right so that everybody could have a better chance of a good life. Perhaps if we truly understood the value of economics we would spend more on getting it right.

Now that we are all more aware, getting it right has become the sole reason governments get elected.

With all modeling, it is all too easy to over simply the model, get things really wrong, but be so sold on the model you're blinded to falicy. This is what I think has happened with economic rationalism, or the market driven economy. (Economists argue on that terminology, so see wikipedia). The true complexity of the economy is terribly over simplified. The ardent believers are blind to the problems of their idealism, and in the end we all suffer.

Animals of the Banking IT Plains

IT is a new industry, and consequently the management of IT is new as well. Everybody needs to know where they fit in; What their job is; What the bounds are; And how what they do, fits in with everybody else.

Project Management (PM) And Managing Complexity?

Well, PM firstly is a methodology for managing complexity. Project Planning, and Risk Management. It also though resides in the realm of human interaction. We've really progressed in the non-human side - planning tools, risk management, and PM frameworks and methodologies, viz. PMBOK, PRINCE2, RUP, CMMI, etc. etc. The human-side though, I think we've got wrong.

Matrix Management

In the days prior to matrix management structures - those of hierarchical management, it was easy - Oh so easy - You simply did what your boss wanted you to do. If you kept him / her happy - bliss; I've managed staff, and I've had employees say this to me. "Matthew, so long as I can know I am OK in your book, that is all that matters to me; I'll be happy". That simply doesn't work in a matrix organisation. I've worked disavowing them of this notion. It can come as a shock that I can think the world of them, but that doesn't count for much if there are those on the other dimension of the matrix don't see it that way, and want to do something about it. You can argue it shouldn't be that way, but you'll understand why it is if you understand the animal world of the modern IT organisation.

Animals of IT

This is how life works. It's political out there. Once you set in place a structure, and function, all else follows. Personalities occupying types of role follow. Pay scales follow. A lot of things follow.

In IT, and in engineering also, we have matrix management, where one dimension of the matrix are [1] Competency hierarchies, and the other side is [2] project prosecution. That is, we have two approaches in conflict.

[1] Competency Hierarchies
Folk who do a function live here. Their line management is here, their career progression, their benefits and entitlements. Well, at least that is how it is presented, but actually, a lot of that is actually determined by the other dimension.

[2] Project Prosecution.
Let me explain a project.
Decide what you want to achieve, plan what resources are going to be required when. Secure the resources. Tell folk to get to it. Track how they are doing regularly. If you are really sophisticated, you'll manage the risks, and the changes to plan also. Many project managers don't do this though. That's it. Pretty simple really.

The key though is that the business interaction is now located here. The business want's the project to, "finish on time, and within budget". Truly staggering recruitment consultants used to put that phrase in job adverts, I guess just to weed out the candidate project managers who put on their CV that they intend not to finish on time or within budget. Those of us who have worked in quality, know that, "to quality objectives", should be added to that list also.

In order to make sense of the ridiculous pay project managers can now attract we give them ever more important names - Program Managers, Segment Managers, Project Directors, Test Directors - but they are just project managers ..

So what's dysfunctional about project management?
Project Management can destroy your organisation, and you'll never know it. The problem is that a strong project management culture can increase the disconnect between those that perform your organisation's function, and the organizational executive management. I've seen it happen.
By creating this additional level of decoupled management, we create a level of management that have these characteristics,

(1) PMs do not understand the functions of the folk they are managing. There are a number of reasons for this, but mainly it goes to the nature of organisational animals. Folk who want to be project managers when they are in school, become project managers as soon as they can after school, and a certain type of person finds success there, and sticks with it. The type of person who is comfortable with managing the domain that they are not skilled in, and generally believe selling is what matters.

(2) PMs are the conduit to the business. The business, and the folk that, year in, year out, do the business, become further apart. This might not seem much of a problem at first, but it is.

Let's look at the interaction between these groups as a conversation.

The Business: "We want what we ask for, on time, and within budget, and to work". Now ideally, the business would build relationships with the functional folk; they would get to know and understand each other better, and the great communications challenge that is at the heart of that throw-away line would be addressed.

The PM: "I'll give you it. You can rely on me to get it for you. Gee these functional folk are hopeless communicators. You can rely on me to get if for you. I'll hire in an expert I know. I'm the one making things happen around here. You need to pay me more. And more. I'll give it you. ...."

I think you can join the dots for the rest. The business end up with less and less interaction with the functional folk, and them with the business. The business end up with no confidence in the functional folk. The functional folk can't get to understand the business any better, and therefore add value. The project managers reign supreme, and their rates of pay soar. If a functional fellow accidentally gets in way of a PM's profile with the business, the PM can, and will, do anything to protect his cash cow.

The functional folk end up not working for the business any longer but for the latest PM, so they actually loose the benefit of being part of a competency - exactly that thing that the matrix management was supposed to bring them.

Eventually, there is little the business can do to correct the situation. Theoretically, they could take the measure of hiring all new PMs that don't have this relationship established, but that's a courageous, and dangerous maneuver, and anyhow, the new project managers will have to be experienced, which means they'll already know how the game works. No improvement there.

In the study of organisational maturity, there is a name given to this situation, Capability Maturity Model Integrated (CMMI) Level 2, "Managed". It says that the business and projects are successful, purely because of the project manager, and the organisation doesn't really have any maturity other than that. Level 3 is "defined", that is the process are defined, well characterised and understood. They are a value to the organisation, regardless of the project management. This really is the beginning of a mature organisation.

I'm considering ways that an organisation can get itself out of this mess, but I'm not ready to publish yet.

References - Entire BLog

Collection of references for blog

1. Business Intelligence: alla Military Intelligence? Making Decisions based on information. Divining the complexity of information collected on the business, in order to make informed, quantitative business decisions.
   
2.