A software engineering process for delivering software
Recently, since Fading importance and the utility of lists, I've been thinking about the top-level approach/process that I use for doing development work, generally.
The main reason was that I was wondering how different other disciplines of development might approach doing work and how much commonality there might be in what I generally do. So I thought it would be interesting to outline my conceptual flow when approaching new work.
A typical strategy that I take to delivering software within the last couple of years, which has focused on being more agile, i.e, exposing and sharing issues more readily and making the strategy design process more collaborative, looks something like this:
|1. Responsibility to deliver is the idea that you are given a task to do and you need to take ownership of getting it done.||
This is less a stage and more a statement of fact really. I wanted to put this down because it's an important realization that as a developer, you need to take much of the initiative to develop a solution, and much of the solution comes from the developer's experience and expertise and knowledge. This is the basis of why a developer trains, is so that they can be given the task to deliver software. It's a personal responsibility first and foremost and this is why you were hired. Of course, as a developer, you will integrate with your team and share, explore and discuss solutions but ultimately you are there to carry out much of the construction work.
It is a psychological element where the responsibility to implement a solution is given to the developer. This usually means that much of the burden of its delivery and issues are thought to be offloaded to the engineer. This inevitably starts the process in the developer's mind of what needs to be done and how they should think about doing it. It also suggests that the developer will be the main implementor of whatever the solution is determined to be and that the success of the solution lies with the developer. This is a fair amount of responsibility and inevitably much of it is assumed.
|2. Interpreting requirements is about understanding what is being asked to be done.||
Generally, this is a process of thinking and rationalizing the requirements and is likely to involve independent research, might extend into social collaboration and discussion if research proves to be less successful etc.
I tend to draw a picture of what is being asked, as this allows me to put down all the elements of the requirements somewhere on the diagram. This helps me ensure that I'm making provision for all the elements in the requirements. My diagram becomes whatever the requirements prompt me to draw. It could be a mind map with arrows, it could be boxes with logical connections etc.
|3. Designing ideas is about thinking about and exploring how the requirements can be realised creatively.||
This would involve modelling ideas that would result in the delivery of the solution by meeting the requirements. This might expand into considering aspects of the design such as the feasibility, architecture, components, quality etc and of requirements and implementation details that are not defined or realised yet.
This process naturally derives from the initial idea that formed in my head as I was interpreting the requirements. This might manifest as a series on conceptual ideas in my mind which I may or may not diagram. This is very much an exploratory thinking process. Typically I extend my initial diagrams with more arrows, boxes or created new diagrams altogether.
|4. Solution design is about selecting a design from the ideas already explored.||
This is about selecting from criteria that makes this design idea better than the rest. It can consider aspects of feasibility, quality, resources.
It is likely to need considering the architecture, concerns and the formulation of the overall design theoretically. It usually considers what is good or time-efficient or other aspects. This will also usually entail considering what technology and technicalities need to be used and be considered.
|5. Technical solution implementation is where the programming happens, and is about using specific technology and implementing it to translate the design into the technical solution that works to deliver the requirements.||
This involves the technicalities of the technologies being used to develop the solution. This stage is likely to be the most transformative and subject to the most change and where most of the time will be spent.
See Strategy for Designing Software Solutions for an illustration of the process
Sharing the responsibility with your group
The stages of this process initially are from the developer's (my) perspective who has been tasked with delivering the software solution. This means that at each step the developer is cognitively involved in rationalising the information and tasks at each step to develop a mental blueprint of the upcoming work that he/she needs to deliver. This means that much of the process is internalised by the individual developer, and any artefacts that are externalised/produced are from the developer such as diagrams, documentation etc is initially produced to aid their own understanding and solidify their development methodology.
Personal artefacts are the outcomes of the process carried out by the developer and can include designs/diagrams, documentation and code
While the process is initially a personal approach initiated and implemented by the developer, many outcomes can be exposed to the group, particularly unknowns and concerns that the developer might have picked up while defining their process.
A problem with personal exposure to the process is that the amount of responsibility and burden that is implicitly on the developer grows as the process is followed and more information is uncovered. This is especially true of discovered unknowns that need to be accounted for or potential concerns or shortcomings that the developer realises not only within the solutions but within their own personal skillset for example.
Holding onto these concerns, unknowns and problems can be psychologically taxing as they weigh down on the developer during subsequent stages of the process such as during Technical solution implementation for example. This is where group exposure to these issues can serve as a relief to these burdens, as it can distribute and share the responsibility of dealing with them, including sourcing potential solutions. This also allows for expectations to be readjusted as many issues are likely to impact the Technical solution implementation stage which is of principal concern for the individual developer as well as the group. This is likely to decrease the pressure that the developer faces when having to deliver the technical solution, as it involves more people (team) and distributes some of the burden such that technical solution implementation can proceed with more clarity with the impact that these issues will have on the technical implementation.
In this way, there needs to be a shared notion of the process where the personal process extends into the group and vice versa, however much of the process is initially heavily centred and focused around the engineer's personal ability to define it. It then needs to be fed back into the group to share and initiate feedback that serves both the developer and the group depending on the developer to deliver the software solution.
Group artefacts are resources that allow for exposure and feedback back to the developer's process and can meetings, design/documentation review, project/goal/strategy/planning tracking and code review.
Generally, I find it useful to produce an artefact that represents my personal outcomes or thinking, such that they can be shared with the team, both for feedback but also for relieving the pressure from dealing with unknowns/uncertainties/issues etc. This includes taking a strategy that keeps track of my progress as outlined in Fading importance and the utility of lists which can help to quantify your efforts throughout the above process.
I would say a large portion of the process is similar and has parity with the typical software development process where requirement gathering proceeds analysis and which leads to design and then ultimately implementation. However, my process presents this slightly differently as more of a personal development process that is followed by an individual contributor (developer) of the overall software development lifecycle of a product or feature.
Also, in the process, there is an emphasis on the initiative that the developer must have when encountering issues, as the developer is often the first person to become aware of them, and failure to expediently expose issues can result in unnecessary pressure in dealing with them. So in this way, this process factors in these personal concerns.
The process tracks individual concerns and processes such as research and the need for understanding and idea modelling. It implies that requirements are already gathered and just need to be correctly understood, or if they are poorly gathered, then that it's the developer's responsibility to determine what the requirements are and then understand them in order to develop a solution to deliver it.
I think much of the business analyst role especially for more agile, smaller teams are now largely deprecated and much of this responsibility is falling on the individual experienced developers.
There is also generally the feeling in my mind that more responsibility is required of developers to deliver end-to-end solutions, and I suspect this is because the complexity of software is difficult to adequately distribute across multiple actors such as an analyst, project manager and other developers.
So more of these tasks are taken on by the individual developer and this requires more skills and better communication skills (which arguably the other roles mentioned previously would be more traditionally good at). However, it's arguably easier to distribute the development tasks to other developers, provided the tasks have been defined - which again is usually the work of another developer if indeed the tasks get distributed at all. In some cases, the developer is responsible for the entire end-to-end software solution due to the difficulty in communicating the complexity of the implementation to others in an adequate time, and so the work takes as long as it takes the developer who understands it to implement.
I think there is a concern in software projects, that due to the complexity of software projects, both theoretically but more problematically at a technical level, it is becoming difficult to speed up development by bringing in new developers as the time to transfer understanding of the complexity is now longer than the time required to deliver the entire solution. So in some cases, solution delivery is not being given strict deadlines but they are progressively being delivered piecemeal as they are being incrementally implemented by the developers who understand the solution/problem/complexity.
In the process outlined, understanding requirements, designing ideas and thinking about how to model and implement them, theoretically and technically is left to the individual developer to satisfy, requiring and drawing knowledge from past experience and training. This suggests that the quality of the developer is being more crucially being recognised as important for their ability to understand, contextualize, rationalize simplify and execute the delivery of solutions in light of the increasing complexities of software development generally.
An interesting discussion that is not touched upon here is how this process differs from the typical game development process, and how either it or this process can lean from each other. For example, is there a similarity to requiring research as part of a task or communicating unknowns to the team, and how is this carried out? I imagine there would be some similarity to non-game development but to what extent or how does it differ?