In August, 2000 Joe Spolsky wrote “The Joel Test: 12 Steps to Better Code“. That was 14 years ago, and since then I’ve developed my own, more up-to-date version based on the original “Joel Test” as well as my own experience.
- Do you have an up to date spec ?
- Do you freeze the spec before developers begin their work ?
- Do you have a published schedule ?
- Do you adjust the schedule based on developer feedback ?
- Do you adjust the schedule based on actual project progress ?
- Do you have a plan for maintenance of your project once it is adopted by users ?
- Do you use a bug tracking system ?
- Do you fix bugs before adding new features ?
- Do your developers have quiet working conditions ?
- Do your developers have time for refactoring and R&D ?
- Are developers allowed to concentrate on their core specialties ?
- Do you use tools developers are productive with ?
- Do new candidates write code before they are hired ?
- Do you have a dedicated QA team ?
- Do your employees use the software they are building ?
1. Do you have an up to date spec ?
I once worked at a company where before I even agreed to work there I was asked to sign an NDA and given a 500 page requirements document with every feature, every workflow, every screen carefully spelled out. When I started on my new job I just opened the spec and started working.
An up to date spec is critical to make sure that nothing is left to interpretation. It must clearly state the requirements and their intent, but leave the implementation details to the developers.
2. Do you freeze the spec before developers begin their work ?
Imagine hiring a builder to build you a house. You come up with a spec for the house, and between you and the builder you agree on what it will look like. The builder acquires materials, hires the right employees, and plans his schedule for the year. However, two weeks after he starts you begin to demand changes to the spec and to the design. In construction industry this results in significant capital cost overruns.
In software engineering it has similar repercussions but somehow we all overlook that. Changing specs and feature creep during the development stage of the project is the number one project killer.
3. Do you have a published schedule ?
Lack of a published schedule results in scope creep and artificial timeframes that create significant software quality risks. Every developer on your team must be able to look at a published schedule. Each and every one of them should be able to go to bed at night knowing that when they wake up in the morning they will work on a feature next on their list. The schedule must be published in a highly visible location and everyone involved with the product should be able to view it.
4. Do you adjust the schedule based on developer feedback ?
Your original schedule may have been based on non-technical goals, such as meeting customer requirements or sales targets. However, it may not always be possible to adhere to your desired timeframe. It is critical to involve your development team in the scheduling process.
There could be technical factors you are not anticipating. For example, your schedule requires that a feature X is delivered in January while feature Y is delivered in February. Your developers, however, feel that feature X and Y both require some preliminary platform level work and either they are both delivered in February, or one is delivered in January and the other in March.
5. Do you adjust the schedule based on actual project progress ?
All deadlines are artificial. Time is a man-made concept. Even the best estimates are usually wrong. Software development is a complex cognitive process that is impossible to estimate with any degree of accuracy. As the project progresses it is important to take measurements and adjust the schedule.
6. Do you have a plan for maintenance of your project once it is adopted by users ?
Who will support and maintain your product once users paid for your product an started using it ? Many startups and small team overlook this crucial aspect of software product lifecycle. There is a difference in skillset between someone who builds the product vs. someone who maintains it. Software rots if not maintained. It also rots if not expanded. If your development team is inundated with support and maintenance tasks your product will not evolve.
7. Do you use a bug tracking system ?
Using an effective bug tracking system goes hand in hand with maintaining an accurate schedule. Just like the schedule must be published in a publicly accessible location, so should the bug tracking system. Requirements, user stories, issues, and bugs must be captured and tracked in such a way that at any given moment a developer can begin work on the next task, and management can track progress.
8. Do you fix bugs before adding new features ?
You don’t want to build new functionality on top of buggy system. That will only amplify the effect of bugs. Fix bugs before adding new features.
9. Do your developers have quiet working conditions ?
Developers should be able to start their day working on the next task, and stay focused on that task until completion.
There are many studies, such as this one, that show the high cost of interrupting knowledge workers. Software development requires a significant level of cognitive effort. In fact, software is probably the most complex system humanity ever developed.
10. Do your developers have time for refactoring and R&D ?
As the software product grows developers find better, more effective ways of accomplishing the functionality. Likewise, to prepare for new features developers must make some changes to the existing code. That process is called refactoring and it is critical to avoid bloat and inefficiencies, not to mention defects.
Furthermore, developers need time to keep up with new developments in their fields. Anything they learn is translated into improvements to software they work on. It is in your organization’s best interest to give them time to improve their skillsets.
Typical ballpark should be that 20% of the time is spent on refactoring, analysis of the design, and R&D.
11. Are developers allowed to concentrate on their core specialties ?
Computer technology is so diverse and complex that expecting developers to have “versatile skillsets” is simply asking for Jack’s of all trades and masters of none.
12. Do you use tools developers are productive with ?
Developers typically do all of their work within a toolkit they selected themselves. I am thinking of Eclipse, NetBeans or IDEA as prime examples. If you ask developers to use tools outside of the area where they do 99% of their work they are unlikely to use those tools effectively.
For example, if you ask developers build design documentation in Microsoft Word they are unlikely to do it effectively and accurately. But if you tell them to build the documentation into JavaDocs and have an automated system build the it upon commits then you are likely to get much better documentation.
13. Do new candidates write code before they are hired ?
When I interview candidates I ask them to fix a bug in an open source project and then call me back and explain to me how they did it. This being 21st century I also ask them if they have a StackOverflow or Github profile with their code samples.
14. Do you have a dedicated QA team ?
If you are serious about building quality software you need quality assurance. You want to treat QA as an engineering effort, not as a testing effort. Let QA engineers build automated testing tools and have staff to test what cannot be automated. Do so continuously, have someone test the application several times a day, as developers make changes to the code.
15. Do your employees use the software they are building ?
Companies should strive to allow their developers use the software they are building. If developers use the software they are building they are likely to build good software. You want your developers to identify with the customers and build tools they themselves will want to use.
This may be difficult in some industries. For instance, someone building a high frequency trading system has little utility for their work for their personal needs. On the other hand, a developer building development tools may find themselves using the fruits of their labor as well.