What to Expect

The following is a broad overview of the software development process, to serve as a guide to readers who may have never been involved in creating a custom software application.

The software development process

Every project, of course, has different requirements and different results. The process of developing custom software, however, tends to follow a few well-defined steps. Smaller projects will typically follow these steps more or less in sequence. Larger projects, on the other hand, are almost always broken down into smaller business modules, each one of which will follow the development sequence independently. Even though they are treated here as discrete steps, often the steps themselves will overlap.

All of the steps are equally important. One of the hallmarks of a quality development effort is that the proper amount of time is spent on the "non-coding" steps. Unfortunately, under tight deadlines, many inexperienced developers will frequently jump right into writing code, without giving proper consideration to gathering requirements, design, and sometimes even testing.

It is important to understand how these phases fit together, and what the roles of the client and the consultant typically are. While every phase of development requires the involvement of both client and consultant to some degree, establishing requirements requires the greatest participation by the client, while the development phase typically involves the least.

There are two common approaches to application development. The more traditional is known as the "waterfall" method of development. In this approach, the steps are tackled one by one, in order, and there is no backtracking. Requirements gathering is followed by analysis, which is followed by program design, and so forth. While in principle, this strict ordering of steps simplifies the process and makes it easier to track one's progress, in practice it is often too restrictive in not allowing for cases in which the requirements are either likely to change, or are poorly defined at the start. It also leaves most of the testing until the end, where any significant problems that are discovered might be too difficult or expensive to fix.

Since the waterfall method has been shown to be unrealistic for many projects, especially larger ones, many (most?) projects today are typically built using some sort of "iterative" or "agile" method of development. In this approach, the steps are still done in order (at least initially), but the project is subdivided into much smaller chunks. Not only is modification of the results of earlier steps allowed, it is expected. For example, program design may demonstrate some essential features that were left out of the requirements, causing the development team to revisit that step. Even with an iterative or agile approach, however, some control must still be maintained over the development process to ensure that the effort still moves forward.

In general, the more agile the process, the shorter the iterations are. In a waterfall project, the inerations are large functional groups, or sometimes the entire application. In an agile project, each iteration includes only a small set of closely related features, or perhaps even a single feature. One of the benefits of agile development is that, one you get past a certain minimal set of functionality, the application is always working and ready-to-ship (though perhaps not with all the eventual features)

Whether waterfall, iterative, or agile, the following is a brief overview of the main tasks involved in any software development project:

Requirements: In this phase, both consultant and client discuss the system to be developed. We determine the minimal set of features to be supported (the "needs"), as well as the ideal system desired (the "wants"). The reason it's important to establish these priorities is that many projects are constrained by time; the client has specific needs that must be met within a specific time frame. This is often one motivation for breaking a large project into multiple development cycles. High priority, mission-critical features can be implemented in earlier phases, while desired but less essential features can be added in a later phase. Consequently, once the requirements have been established and prioritized, an initial timetable is determined.; Often the starting point in establishing requirements is a careful analysis of the existing process which is to be automated. By examining the existing process, we can fit the design to the way your business actually operates. Of course, this examination also opens the possibility of modifying the business process to take advantage of the way computers handle information. For example, you would only need to enter data once, even though in a manual process it may need to be copied several times. The key here is to remain flexible, and look for opportunities to streamline certain tasks, without forcing you into any specific procedures.; One of the most powerful tools used in establishing requirements, particularly user interface requirements, is prototyping. What that means is that the consultant will create a dummy application that looks like the final product, but without any of the business logic built in. This prototype is easy to build, and can usually be done in a few days, even for larger systems. Its purpose is to allow the user to see what the final product will actually look like, and get a sense of how much data will fit on one window, how that data is organized, what steps are required to perform specific tasks, and so on. What that means is that usability issues are addressed early in the development process, while they're still easy to correct.
Analysis & Design: One of the most important steps, even though it's frequently overlooked. It actually consists of two steps. The first is to establish a high-level design that specifies what the different parts of the system will be, and how they interrelate. During this phase, decisions are made regarding what hardware and operating systems are required, as well as what software tools will be used. User interface standards are established based on the prototype discussed above, so that every window has a consistent 'look-and-feel' in its presentation, and is consequently easier to use.; The second part is a detailed specification of each business module, window, and function within the system. A detailed database design, or schema, is also established at this point. As with prototyping, performing a detailed design allows changes to be made early, while it's still relatively easy. Done properly, this serves as the foundation for the technical documentation of the system. It also serves as a blueprint to guide the development phase. At this point, a firm timetable can usually be established.; Note the distinctions between requirements gathering, analysis and design. Requirements gathering refers to the task of identifying the business problem or need at hand. Analysis refers to the task of understanding the business problem at hand. Design, on the other hand, refers to the process of planning the solution to that problem.
Development: This phase is mostly self-explanatory, although it usually comprises the majority of the project life cycle. Taking the design established in the previous phase, the application itself is then built. As mentioned earlier, often the system is built in multiple phases so that critical functionality can be deployed as early as possible.
Testing: Even though it's listed as a separate phase, the bulk of the testing actually occurs during the development phase. This first part is commonly known as unit testing. Its goal is to make sure that the individual components of the system work well, both separately and in conjunction with other parts of the system. If done poorly, the user will later experience problems such as system crashes, or poor performance. Done well, the user may not even be aware of this form of testing at all.; The other type of testing is known as system testing. Its goal is to make sure that the original requirements have been met, that the business rules embodied in the system are correct, and to verify that the system works as a coherent whole. This stage of testing is done in close coordination with the eventual users of the system. This is another reason why it's important to have the participation of the end users in gathering requirements. Typically, a limited-scale deployment of the hardware and software is done in order to facilitate the testing. Often, a new application will replace an existing one; in such a case, testing often includes running both systems in parallel for a defined period of time.
Documentation: This is probably the most frequently overlooked step. Documentation serves two purposes. Technical documentation is used as a guide for future developers to make changes to a system months or even years later. Even the developer who designed and wrote most of a system would be hard-pressed to remember every detail of its implementation after a period of time. User documentation, on the other hand, is the guide to using the system, as well as making the most of its features. User documentation may come in different forms, from printed manuals to online help files. While one of the goals of effective user interface design is to minimize the need for user documentation by making the system easy and intuitive to use, proper documentation is nonetheless important in training both current and new staff in the use of the system.
Deployment: This is where it all comes together. Hardware is installed, and the network configuration is established. Depending on the design of the system, database, application, and web servers are installed on dedicated server machines. Existing data, if any, is converted to the new system. The finished application is then installed, and final testing is done to make sure that all of the pieces of the system are working correctly in concert with one another.