The SharePoint Governance Puzzle

The SharePoint Governance Puzzle

If you have been using or managing SharePoint then you would know how quickly SharePoint content, customisations and their growth, if not properly governed, can get out of control. Today most of the SharePoint customers have realised this and have SharePoint governance plan and policies defined for their SharePoint deployments. A governance plan could just be a brief document focusing on what users can and cannot do or a detailed framework governing every bit from SharePoint development to usage and maintenance.

However, a large number of organisations are still struggling in having effective SharePoint Governance in place despite of having a well-defined governance plan. To understand the reasons behind this issue we have to remember that a governance plan is not the solution but a part of the solution. Often organisations' focus is on defining a detailed and comprehensive governance plan but that focus gets shifted away when it comes to implementing the plan. Once a governance plan is created it is somehow assumed that it is now up to IT support to ensure the processes and policies defined in the plan are enforced and implemented.

So let's first take a look at the key implementation challenges and then we will see what can be done to address them.

Implementation Challenges

A. The implementation deserves respect, treat it just like another project!

One of the reasons why the implementation doesn't get due attention is the absence of some tangible outcomes. Implementing a plan and associated policies is not like delivering a system. Furthermore it is not that straightforward to measure the outcomes and see the associated benefits. Therefore often interest in rolling out the governance plan dies soon after its creation. As a result governance 
documents are put somewhere on the same SharePoint site that is being governed and forgotten.

B. Yes, it does require (a little bit) of efforts and resources!

Another challenge is the availability and allocation of the efforts and resources required to enforce the governance policies with consistency. This gets further complicated when we recall the fundamental principle behind existence of SharePoint i.e. user empowerment. This can give rise to uncontrolled SharePoint growth where the balance of end-user power and IT control is not right for the environment. In other words SharePoint governance is lot different and demanding than managing, say your Active Directory. This makes it difficult for IT to enforce the policies without hindering user productivity (and annoying them).

C. If they don't know it then they won't do it!

Finally, enforcing governance does not merely require flicking some switches. The first step in implementing any set of governance policies is communicating them to the users and support staff. However the reality is that users often do not have the time and interest to read lengthy governance documents. Being humans (and as SharePoint users) they do not like tightly controlled environments either, they are needed to be convinced that the controls you are putting place will eventually work for them. An unhappy and ill-informed user may, instead, negatively see the governance policies as a roadblock towards being productive.

The secret behind a successful governance plan implementation

Well, there is no secret here really. In summary, it requires the IT management taking the implementation seriously (just like any other project) and seeing the benefits; doing effective user training and communication; and investing in automation to assist the IT teams in enforcing the governance policies.

User Communication and Training

We know very well that no one loves to read lengthy documents. Furthermore, reading text is one thing and remembering the content is another. So post governance framework creation the next challenge is to communicate it to the users (end user and support staff) while ensuring it is not too difficult for them for them to understand and remember all the key points.

One technique that can help here is creating summarised posters or cheat-sheets for specific group of users such as SharePoint site members, owners, administrators etc. You can combine the information with any visualisation techniques that you feel may be effective, like charts, tables and even comics. See one such an example below

It is also quite helpful (I would rather say necessary) to integrate the governance plan 'knowhow' into HR & IT processes like employees' induction, role change and termination procedures. This will ensure things happen when they are required to.

The next big thing is … Automation
To help with enforcing governance policies without creating a lot of overhead for IT support and end-users, you can also get assistance from some SharePoint governance tools. There are quite a few third party tools available for this purpose. These tools can provide

• finer control in managing the SharePoint environment;
• powerful growth forecasting and management;
• comprehensive and centralized security control and permission monitoring;
• provisioning workflows;
• critical alerts and notifications etc.

We can divide the tools into two categories i.e. strategic and tactical. The basis of this subdivision is the product feature set, associated costs and resources required to implement them.

[I had another blogpost written on SharePoint Health Check which you can check here]

Tactical Tools    

Tactical tools are low cost options that enable the support teams to analyse and understand their SharePoint implementation and assess its health with reference to the governance plan and policies.

1. SharePoint Documentation Kit (SPDocKit): A windows application that can run standalone or be installed on the server. For more information check the tool here SPDocKit

• generates detailed farm documentation on the target SharePoint environment
• compares SharePoint farm configuration against best practices
• can take snapshot of the environment and allows the support to compare with other SharePoint farms or the same farm but a different point in time

2. The SharePoint Diagram Tool: This is a quite handy tool that enables you to check if your SharePoint sites are having some kind of mushroom growth or flourishing like a beautiful well-maintained garden. It reverse engineers your site structure and let you visualise it in tree structure form. The tool can generates its output in multiple formats and you can then use MS Excel, a browser or Visual studio to render the site structure. Regular generation and analysis of site structure diagrams can be added to IT support processes for detecting any major violations of site-structure related governance policies. http://realworldsa.blogspot.com.au/2010/12/new-tool-for-sharepoint-2007-and-2010.html

Advanced Tools

1. ControlPoint by Metalogix: Metalogix offers a suite of tools for migrating and managing SharePoint content and that includes ControlPoint. ControlPoint is a well-refined product with an extensive set of features. It includes all the key components such as provisioning workflows, content growth monitoring, user action reporting and site stats; governance policy and permission management etc. For more information see http://www.metalogix.com/Products/ControlPoint.aspx     

2. AvePoint Docave Governance Automation: A solid product for SharePoint content migration and automating governance workflows. It can be seen as a strategic investment i.e. having AvePoint as the provider of some key SharePoint add-ons. They offer a range of SharePoint products, both for SharePoint in cloud and on-premise. However the governance module mainly focuses on SharePoint sites and site objects provisioning and deletion workflows. You will need to buy additional module(s) for end-to-end monitoring and reporting purposes. http://www.avepoint.com/products/sharepoint-governance/

3. Sharegate - SharePoint migration and Management Tool: An easy to use product offering both content migration and governance tools in one product. It allows the SharePoint admin to manage security settings, monitor environments' growth, get rid of unused & obsolete content and ensure that SharePoint meets the organisational standards. However, if you do not have SharePoint content migration needs then you cannot buy the governance module as a standalone product. Find more information at http://en.share-gate.com/

4. Acceleratio Governance Kit for Office 365: This is a low cost cloud based office 365 governance tool (they do not have a version for SharePoint on-premise). They also have another product SharePoint Documentation Kit (described in the 'Tactical Tools' section previously) that can help with reporting and site structure analysis. With this governance kit administrators can setup and configure SharePoint Online rules. The rules then can be applied to a specific site, list or library. It can generate detailed reports. Find more information at http://acceleratio.net/products/governance-toolkit-for-office-365/features/

I conclude here by emphasizing that first and foremost the implementation of a SharePoint governance plan should be taken and treated like a project. This approach ensures that the SharePoint governance policies do not just stay within the boundaries of a document but they do come out into action and reward everyone.

I have also published this post on my own blog The SharePoint Governance Puzzle

Writing Software Requirements Specifications | A Technical Communication Community

I like to share this very nice article on writing requirement specifications, reading that was like a taking a step back and think again

Writing Software Requirements Specifications A Technical Communication Community: "Writing Software Requirements Specifications
by Donn Le Vie, Jr.
Here's the scenario: You're finishing up your latest HTML Help project...no more late nights or weekends...back to a 'normal' 50-hour work week. That's when the development team lead strolls into your office and says she just got your manager's okay for you to help the development team 'put together the functional requirements specification template for the next major project.'............."

Software development - A to Z

A picture is worth a thousand words

Bridging the gap: Business and IT

We know the people who give requirements called 'Business' and the people who develop the requirements the 'Developers'. Also we know that there is another group of people who work as coordinators among these two groups and called 'Analysts'.

The challenge,

For analysts the major challenge is to make both 'business' and 'developers' happy. They deal with the people who want to stick with their guns and hard to change their opinions. Business just believe in getting whatever they want 'somehow' and Developer whatever they 'understand' to deliver is to deliver with the technology they know the best. The analysts have to master the art of communicating to both the groups in their own language.

What happens in the real world,

Analysts might come from one of two backgrounds; IT developers with strong communication and problem solving skills or Business people with strong management and understanding skills.

A successful analyst would be the person who attempts to see the picture from the other side.
Limiting the focus to the areas they know best do not deliver the results.

For instance, an analyst with background in business may attempt to force the development team to cut the crap ("methodology and processes") and deliver fast. On the other hand an analyst with expertise in IT may try to focus more on technical solution (tools and technology and methodology) than functional solution (the process and achieving the value), stressing the quality of the software.

recipe for success,

DOs

1. Realistic
2. Quality of software and processes does matter
3. Business people needs solution to their business problems
4. Developers want clarity so remove ambiguity from requirements as much as possible
5. Business is more interested in functional solution then technical design so give them the only part they are interested in.
6. Speedy communication is the key to success
7. For everything, coming from developer or business, ask again and again 'Does it make sense?'
8. Be flexible but Quality Does Matter!
9. Learn how to understand people, deal with each person (technical or business) with the way they should be
10. Understand the environment, the people, the context and the people. Every place is different so modify your approach but always make it goal driven.
11. Be agile, embrace changes and updates
    

DON'Ts

1. Avoid attempting to be 100% clear on requirements , business will never (and they can't) give you. Make assumptions, take decisions and move forward with an acceptable margin of error
2. - Avoid stressing too much on technical quality when it will deliver no value, making process too difficult to follow and creating a pile of documentation which no one would be interested in reading.
3. Don't rush to get something done with half cooked requirement. you might have to scrap the whole work and do it again. In IT delivering a software with 95% functionality working means having bugs around 5% mark, too high!.
4. Get out of your shell, your background (IT or business) is your strength, don't make it a constraint

Just a link,

http://www.requirementssolutions.com/Business_Analysis_Skills_Test.html

SOA Maturity Model: Deep inside SOA

Again I have a book to recommend for reading if you need to understand/revise the philosophy of SOA and get some practical guidelines on implementing Service Oriented Architecture in your organisation.

SOA concept is quite old now but mostly we face challenges in implementing it successfully.

The basic issue that we face is when to start and where to stop or where to start and when to stop.

Some people get it confused with web services and they will build a whole stack of web services, every function within an application will be exposed as a web service, some will redesign the applications from scratch using some SOA framework and some will buy expensive tools to expose business functions within some legacy existing applications .

No doubt there is a lot of confusion out there. Even some people who have a good understanding of SOA find it really difficult to map it to a real business problem/environment because every place is different and SOA initiative required to be largely context driven.

For me, the major challenge is to decide if some project qualifies for SOA or not. The major issue is the time that we invest in developing SOA ready applications. I have seen the places where development teams follow strict standards, processes and develop highly reusable applications (or components) at the cost of slow response time to new business requests and changes. Sometimes the management gets so frustrated (as I witness a number of times) they themselves ask to compromise the quality and just deliver the solution because that's what the business demands. Sometimes it works (where we really need a throw away application) but many times it backfires (where we need a stable system, an LOB application).

So how can we achieve success with SOA?

The first basic thing is that the developers alone can not bring SOA revolution in an organisation. The SOA implementation must occur both in functional areas (business units/departments) as well as in technical areas. Business processes indicates what "functional services" can become the candidates of reusability and then IT attempts to model and deliver the systems that would map the interactions in real business world. Also studying business models will help understanding what services we might need and what not.

the book,

It is difficult to explain all here, therefore, I recommend reading this book SOA in the Real World which is a nice book and starts from the basics, takes you through the practical steps for a successful SOA implementation and talks about Enterprise Service Oriented Maturity Model (ESOMM) in good details. This book is free to download from Microsoft.

Happy Reading.

A Cool Utility:

UltraExplorer is designed to be the ultimate File Manager for Microsoft Windows.

http://www.download.com/UltraExplorer/3000-2248_4-10702384.html?tag=dl-blog

Deep inside Agile Development

A nice article,

I have just read a fantastic article written by Ted Neward on pragmatic agile development, from an architectural prespective. I must say he beautifully put together the philosphy of agile development, core issues that we face and what exactly it means by agile development when it comes to delivery of the solution. read it here

http://msdn2.microsoft.com/en-au/architecture/bb892770.aspx


and I love his conclusion,

"Call it what you will, the basic keys to successful software remain the same: good people, whether they’re developers or managers; good process, whether it’s lightweight or rigorous; a good product, whether it’s detailed in a document or sketched on 3x5 cards; and good technology, whether it’s COBOL or something a bit more modern. Many shops do one thing right, some do two or three, but the truly spectacular successes get all four right, and the results…well, they speak for themselves, and need no buzzword to define them."

and the book,

the article reminded me of the book, Rapid Development (well how could I forget that book!) that laid the fondation stone of my software engineering concepts when I studied it at Uni in 1997. Its third chapter "Classical Mistakes" is fantastic and the list should be put on the walls of a software house as we tend to forget and commit such classical mistakes again and again. http://stevemcconnell.com/rdenum.htm

happy reading

SOA everywhere - Delivery nowhere

Well, the title of this post is bit exaggerated, SOA is definitely good but implementation is hard and a number of times we end up in creating a mess, took ages to develop but failed to deliver success.

What is good about SOA?

As we know when we say SOA we mean that we need something that would be driven from business model and functions, would reuse existing functionalities with minimal impact on existing applications and making them interoperable with other systems so that we could save time in redevelopment in future.

Also when we develop new systems for future we attempt to make them 'SOA Ready' i.e. designed in a way that any business function can be made available to other systems/consumers without or minimal coding. Of course the idea is fascinating and it does work.

so what doesn't work?

Here the problem I want to address is that having a vision of SOA when technical people sit together and design a framework + lifecycle for the entire business apps and then if they force all development work to follow the same framework for all new developments in order to have consistent development practices, SOA ready applications, potential long term time saving in new developments, etc, there are some serious issues arise from application delivery perspective, particularly slow delivery of solution due to huge investment of efforts upfront for making applications 'SOA Ready'.

But do we really get all those benefits in future? The answer is context driven i.e. it may be or not depending upon the nature of business and development projects. But what I have seen a number of times you can not apply one solution to all your problems.

Take this example, you have got a very nice framework, a framework labelled as SOA based framework, with all nice components, this block, that service, this interface, that data contract and this and that etc. Now you have got a small application to develop for a small group of users in your environment. Would you go for an SOA ready application?

The mistake, in my opinion, people make is that they will go straight after a SOA based architecture while in principle the application is not a right fit for SOA. They do it becuase it is the standard architecture and practice of the organisation now. As a result, developers spend a lot of time in fulfilling framework requirements by developing interfaces, layers, services etc with very less probability of any reusability in future.

so what to do?

In my opinion whenever we design a SOA ready framework we have to consider following points

1. First SOA based architecture should not be a mandatory for all new applications to follow

2. Every application will have to be evaluated to see if there is any real benefit in developing it as an SOA ready application. This depends a lot on how business processes work and not the technology. If the new application is for a small group of users, who are working in isolation with less probability of exposing their functions as services, then we can use very simple architecture and forget complex SOA based architecture.

3. Also, the framework should have pre-defined cut down versions for medium and small applications with lesser number of layers/services/interfaces in order to reduce development overheads. The application will be judged and right model should be picked and applied

4. If using agile methodology, then future reusability can be left over to refactoring. What I mean is that instead of investing time today, leave it to the time when it is required to expose its services and at that time refactor the application and deploy it as an SOA service.

5. As a general rule core systems should be strong candidates of SOA style development. For example, customers database, core sales operations, main financial systems etc. While the systems like some feedback capturing application, some monthly computing application with short expected life span (2-4 years) etc are bad candidates and should be developed in a quick and easy way.

in summary,

We should focus more on delivery without investing too much time in provisions for 'future reusability scenarios'. For instance, once it happened to me that my team put a lot of effort in developing an application with SOA based architecture and ending up with the situation where business users discarded the application, never went live as the business owner of the application had left the organisation and the new owner did not see any real benefit in the application. All our efforts for a 'highly flexible and reusable service based SOA style' application go in vein.

question to ask yourself,

So next time you get an application to develop, ask yourself, does it really need to be developed as an SOA application?

The fear factor

Just a couple of quotes to start with

"Fear makes the wolf bigger than he is."---- German Proverb "

“Each time we face our fear, we gain strength, courage, and confidence in the doing.”

The above quotes just talk about fear factor in general, but I will go and talk about fear factor in software development.

You fear => You Fail,

This is an observation that I have made consistently in my career, if you fear then you will fail, and recently I have observed the same again, making me write this blog. Doesn' t matter whatever technical skills we have got but too much fear of uncertainty can spoil our hard work and progress.


By nature we tend to avoid taking risks, we attempt to follow a path with higher level of certainty and when it comes to software development then we translate this in having well defined requirements, using tested and proven technology, following well practised and adopted development life cycles etc. Particularly if we have spent working years and years in an environment following typical water fall life cycle models. So when we move from such an environment to a dynamic environment where all variables 'requirements', 'technology', 'timelines' and 'resources' continuously changing, we start fearing

- I can't suggest this technology, what if we fail?
- I can't develop this feature, what if doesn't provide value to business
- I have to commit to the deadline provided by my manager, otherwise....
- I have to provide all what my users/business ask, otherwise....
- I have to follow the current practices and procedures religiously and rigorously, otherwise...
- I have to do all documentation
- I am uncertain about requirements, I have to wait and get all clarifications
- I have to save my back
- I could be kicked out if I fail

the list goes on


Agilists don't fear,


One of the core characteristic of some agile development life cycle is that we try to get rid of the fear factor i.e. Agilisits don't fear (but they are not stupid too!). They take the decision early on, they believe that the total cost of pending your decisions too long will be higher than making a wrong decision (among other correct decisions) and rectifying it later. To illustrate this just read this fictitious challenge


A fictitious challenge,

Task: You have to reach your destination 'City A' as soon as possible. You have got two options.

1. Take the safe road that will take 24 hours to get you there.
2. Take a short off road trip, that will take you there in 4 hours.

What would you do?

A risk avoiding person will take the first option. Spending more time and money in reaching the destination, fearing the second option will have unforeseen risks and he will face a lot of trouble

An agilist will take the option 2. BUT as I said Agilists (supposedly and should be) not stupid. He will get a 4WD, with spare tyres, GPS device and a satellite phone, other tools and equipment necessary to rescue himself if something goes wrong and then begin the journey.

well, hmmm...,

You don't agree?? What if this and what if that...? Well it just a fictitious scenario and yes if reaching the destination is a 'do or die' sort of thing I myself will take option 1. But in general I should go for option 2.


The agilist might fail reaching the destination but if he takes such decision 10 times (and are used to taking such challenges and well trained on using his tools) he will probably reach the destination 8-9 times and when he fails to reach the destination he might have to come back and take the safe route, 1-2 times. The point is that the overall cost of failures would be less than the cost of late decisions and lengthy processing times.

So how we map it to software development?

In typical agile development we train ourselves

on techniques like

- User stories
- Automated unit testing
- Continuous integration
- Refactoring
- Iterative development
- Just enough documentation
- Strong communication
-etc etc

using tools

- Nunit/Junit
- Cruise control
- Codegenerators like Codesmith
- Resharper
- Task management tools that support iterative development (JIRA, Gemini, MS TFS)
- communication tools
- etc etc


The message,

The message I want to convey is that when you develop software in today's world, accept challenges and prepare yourself, take initiatives and get ready to mitigate the risks. Take stand on technical and quality grounds. Ask why this way and not that way. don't accept something in the environment just because it was defined in the past.

Challenge yours and others decisions. If you BELIEVE in something then stick to it and make other people change their course of action.

It is better to be kicked out of you organisation on taking stand on quality rather than being kicked out because of not taking initiative and appearing as a dumb person.

Agile Toolbox

How many times in our environment do we find the developers not following the right processes? Sometimes it creates serious issues, affects the software quality, adds to team frustration etc.

So beside designing and developing development processes it is equally important to ensure they processes have been adopted and practised regularly. There are few tools available that let you enforce the processes in your environment. The objective of these tools is too provide one window solution where team members can collaborate and follow the (customisable ) life cycle model, project managers can manage, developer and tester can work together, and that can be used as a central repository of the documents etc.

1. RallyDev http://www.rallydev.com/
A very nice tool, offers ajax enabled dashboard/interface to plan and monitor releases and iteration, allows you to follow either some Agile or RUP development methodology and automatically fetches the unit test results (coming from some automated unit testing tool). a demo can be viewed on http://www.rallydev.com/Rally_14_logofix.html

2. Collabnet : http://www.collab.net/products/enterprise_edition/.
Free for 15 users, claims to have more than 1 million users and they have recently acquired sourceforge. It primarily targets the companies with team members located at different geographical locations but of course it can be used for a consulting or internal IT environment where multiple projects run in parallel. You can check their live demos on their web site

3. VisionOne: http://www.versionone.com/
Another product similar to RallyDev but interface is not that impressive. check the demo here
http://www.versionone.com/flashtour/VersionOne_Product_Tour.htm. Also they offer a tool they call 'AgileToolEvaluator' which basically is a checklist of the items you should be looking for in an Agile tool. though it is good but again it is from versionone so the checklist lists the items that versionone tool supports. So you can use it but also consider your environment specific requirements.

There is another buyer's guide from 'Forrester' which costs US$279 and help you in identifying your requirements and what you can find out of the tools available. find it there http://www.forrester.com/Research/Document/Excerpt/0,7211,34977,00.html

There is another good article http://alistair.cockburn.us/index.php/What_the_agile_toolbox_contains

One more tool, available for free, but for those who are following XP. http://xplanner.org/

Next time (or sometime in future) I will talk about independent tools and utilities and how those could be used in a combination to follow some agile process.

Step by step

Last time I discussed the key characteristics of software process improvement exercise. This time I will be focusing on what steps we can take to develop or refine our methodology.

Again you are working as a consultant, assigned to refine or develop the development methodology for your client.

1. Understand the environment: Identify typical projects, team structure, existing skills, past history of project delivery, nature of business/clients, create a picture of existing environment.

2. Identify the needs and objectives: Find what your client expects from you to deliver but be careful it is quite possible they themselves do not exactly know what the need and may ask you to deliver something irrelevant. At higher level, with the help of your understanding of their existing environment and prcoesses, identify their main pain points and areas where improvement can be achieved.

3. Build a Team: You may be required to work alone and on your own but always try to have some team on board for software process improvement exercise. Though you might be working on the assignment primarily but a team should be there to review and verify your work on regular basis, say weekly or fortnightly. Such a team should include people from Technical Management, Project Management, Development and Support. If you can't build such a team then you would have to do your assignment in multiple iterations i.e. developing the process, putting it in front of development team & management for review, getting feedback, making changes, feedback, changes, feedback........ though this may result in lot of rework and deadline slips.

4. Develop a roadmap: Develop a plan, identify major deliverables and milestones. Do not attempt to come up with a schedule with fixed dates, talk in terms of weeks or month. Keep enough room in the schedule to accommodate any additional work.

5. Explore the existing practices: Start going into details of the existing process, find why do they fail, attempt to identify repetitive problems and their causes, find how the processes were developed, how religiously existing processes are followed, if not followed then why not, what developers like and hate most, where are the conflicts of interest among different Teams. Draw detailed process flows.

6. Research Agile Methodologies: Having problems, challenges and needs known, attempt to find where agile techniques could help you. Do not attempt to adopt some agile methodology completely, only look for the solutions that would be able to help you. You may get benefits from techniques like capturing requirements through user stories, iterative development, continuous integration and automated deployment, automated unit testing, unit test first, pair programming, stand-up meetings each day, having client representative on board to assist development team during development etc. Be mindful that some techniques may be dependent on other techniques to be adopted as well.

7. Apply the techniques and refine the process: Attempt to see (in real world) how the new techniques, activities or artifacts would help you in overcoming the existing problems. Dry run the process, think about all possible scenarios (small/big projects/team etc), refine the process flow, add/delete activates, artifacts, again & again challenge yourself by asking why (& why & why). Be mindful of jargon and terminologies, do not unnecessarily attempt to replace the existing ones with some new ones when there would be no clear benefit.

8. Documenting: Once your newly defined processes look stabilising (after reviews and reviews), start documenting it. Document the entire process, phases and activities. You target audience will be a new person/developer joining the team without any prior experience in processes based development. For each activity, describe the objective, input/output, guidelines to perform that activity, success criteria and any support documents to be used to perform that activity. Create the templates and more importantly samples of the artefacts. Samples are the quicker way of communicating a process or standard and provide some reusability.

9. Present it to the entire development team and review with them. Don't be afraid of making it flexible and giving some room to the developer to manoeuvre, but of course not at the cost of quality and violating the process. For example, you can let them choose use cases over user stories for some time, or allowing developing unit test cases after coding (but before checking in the code).

10. Quick reference docs: Develop quick reference or activity specific documents, ideally one page only for each doc so that developer could stick them on their desks. It may include development principles (security, performance etc), Unit Testing Guide, Source control guide, coding standards, company specific application frameworks and architectures.

11. Publish it: Publish it and make it accessible to the entire development and technical teams. Here one tip, try 'Save as html' feature of Microsoft Visio. You can very quickly convert your Visio diagrams into web pages and link up all process documents to the html pages, making it a single point of entry for the developers.

12. Run a Pilot: Job is not finished yet. Before asking the teams and developers to start following the new process, identify some less risky pilot project and use the new methodology. Keep on tuning the documented process with any improvements identified. Share the results with the developers.

13. Hand-it-over to development team: Perform training sessions; while planning the project work consider any extra overheads due to new process., have someone takes the ownership of the process and monitoring closely in the initial months to identify any issues and help the development teams.

See you next time.

Software Process Improvement - Critical Chracterstics

When you start work on improving your software development methodology, you feel that you need some framework or process to carry that task. Here I will try to put some guidelines about getting that exercise done.

Let's take a scenario; your assignment is to develop and deliver a development methodology framework for your client (or your team).

Your assignment should be,

1. Focused and Result oriented i.e. You want to see the results and are not trying to fill the file folders with heaps of good looking documents and diagrams. You know your objectives well.

2. Iterative process with milestones: You have a plan in place. You understand that you can not achieve all in one go and you have to make the improvements in multiple iterations.

3. High level of development team participation: You are not working alone in isolation. You are expected to incorporate the development team's feedback into your work as they are the real people who feel the pain.

4. Only necessary process documents and less garbage: Again, you have to be very critical of the documents that you suggest to be developed as part of the development process. First, only necessary documents, second, the documents should be precise and to the point. Prefer tables, charts over lengthy paragraphs to communicate the information.

5. Adopting best practices, only if applicable: No silver bullet syndrome. Best practices might not be appropriate for your environment no matter how 'hot' they are in technology world. Adopt them if you see that they will deliver the results, otherwise don't waste time.

6. Living documents: This is very important. Your process and practices will not always remain the same so should be the process documents. The process documents should reflect the current picture of the development environment. Ideally (yes ideally!), whatever written in the docs is what we practice.

7. Pilot implementation of newly developed methodology: Pilot implementation should be the part of the methodology development process. A pilot implementation would offer you a chance to test your recommended processes and practices. It will verify how practical those recommendations are in real world. Also it would help the developers in adopting the processes as they will have an end to end process sample to follow.

9. Communication Plan: Communicating the newly developed processes should be the part of the process improvement exercise.


Next time I will discuss specific steps that you can take to come up with some practical development methodology and framework.

Development is one thing, publishing is another

You may fail a methodology if you fail to publish it.

Success of some newly developed or adopted development methodology largely depends on how successfully it is communicated to the development team. For example, creating heavy documents and putting them on some network folders won't guarantee that the methodology would be followed and understood well by the development team.

There are few techniques,

-Publish the methodology on some local portal/intranet. The data should be highly visible, linked and searchable by the team. Preferably the information should be published in html format rather than in the word docs.

-Run training sessions for newly hired developers

-Prepare summary documents that could stick on the developer desks for quick lookup (like coding standards, process flows etc)

A lovely scenario
" a newly hired developer starts working on software design and need to know how to prepare design specifications document?, knowing a simple link http://ourportal/DevMethodolgoy, clicks there, opens the page, types in the search box 'design specifications', the result pages displays the design process with detailed guidelines about the activities, process flows, templates, roles and responsibilities, sample documents and archive of previously developed design documents. In few minutes the developer knows exactly what needs to be done and how"

Be aware of methodology!

Real world story:

These days I spend my weekends in working with my builder to build my dream home. Last weekend he asked me
" This is the thermal specification sheet and you need to fill in and get back to me by next weekend. In it you will be specifying wheather you prefer dark, medium or light colors for your external roof and walls."

I was a bit surprised" Well, my understanding was to decide about the colors at a later stage as we are still 4 months away from starting construction. "

"First you need to decide about intensity not actual colors, and second it is a council's requirement for approving the plans so we can't do much"

"What if I change my mind later on?" I was a bit confused and thought how could I decide about intensity without deciding the colors

" I do not suggest this as then we will have to go back to Council and it will require few weeks to get the change approved and it will cost you a lot" Builder was quite straightforward

"hmm, ok, so if this is the process then it means I will have to do it"

Next week I faxed the specification sheet back to the builder and made up my mind that this is it, good or bad , I will have to live with it.

Moral of the story:

That experience made me think about the way we develop software (with whatever agile or fragile methodology). In my job, I play the role of the builder for my clients. The thing that I learn from this is that beside training your team on the develpoment methodology it is equally important that you do the same for your client. Clients or users come to us with the dreams of the software and being human they keep on changing their minds. It is our job to show them the real world picture and we shouldn't be too late in doing that.

In the above example I might not agree with the process (deciding intensity without deciding colors!) but still I accepted it as a fact and worked to minimise the impact.

In software world, having some agile methodology to cater for the on going changes is good but there are some limits. We need to clearly communicate to our client/user the trade-offs, flexibility/rigidness of the processes, timings of the key decisions , impact of making changes at later stage, associated costs, etc.

A number of times our relationship with our clietns turn bitter only becuase we fail to communicate all of that to them in timely manner and thus turn their dream into a nightmare. I have experienced that it becomes easier to work with users if we had already informed them about what they could do and couldn't.

Dreaming!
I feel confident now that I will be able to get my home built on time and with in budget becuase of having a straight forward builder who is making things crystal clear to me (well, I hope :) )