Software exists all around us - we interact with it and it affects us all and to a large extent this is due to the social and economic factors when dealing with software, it can cost you a lot of money to make but it can also save you money. It can also cost you money to maintain and to deal with problems and failures in the software.
It can also make you a lot of money.
Here I'd like to spend some time speculating about software, why I'm 'in it', what it means, what its actually doing for us, what its all about and what happens when it fails - the wider and bigger picture. I also share a few viewpoints I have on the role software developers will have moving forward.
Software does fail and controlling it is a study/discipline by itself – this touches of software quality and making the effort to ensure its high, planning so that it's delivered etc.
Software arguably helps businesses the most; businesses have quickly identified that software is an extraordinary tool which can be seen as a replacement for manual jobs people do in many ways. However, it can’t replace the fundamental ability of people which is to think and react to change, circumstance etc. I'll touch on how change is such a fundamental concept in software.
There are multiple approaches to developing software and bizarrely the perceived benefits of software systems are often difficult to actually realise(ie it costs a lot of money to develop and the software delivered doesn't meet many expectations), in part, this due to difficulties to predict and plan how the outcome of the design and implementation issues generally. It also has to do with not actually realising what the real problem is that the software is trying to solve. More about requirements for commisioning a software project later.
This hints that perhaps the development process might be flawed or that software developers don't know how to develop software? As you'll see, there is more to this than this and there are things lurking in the shadows that actually cause software developers to deliver software which is not what is required.
So, often software systems fail to deliver expectations. I wonder why? This is usually because of the prevalence of change that occurs through the software development lifecycle and the emergence of this change but also because to lack of visibility of requirements or lack of establishment of them at all!
With respects to change, models like the waterfall method tries to limit change to solve this problem of unexpectancy(my word) which initially makes sense, however, it conflicts with an eventuality which is people’s minds actually DO change and ideas actually DO become better as time goes on or that the understanding of problems DO in fact change with reflection over time. This usually occurs during an already-established course of a software development endeavour. Change is inevitable and its change that de-rails plans usually.
Investment in information technology can be a bottomless pit and reap little rewards if not correctly monitored from inception, design and planning all the way to delivery and maintenance.
You need to be on top of managing and reasoning about change (A dangerous miracle) – being agile in reacting to it and applying its repercussions positively to the development objectives at hand during the development endeavour. It’s a hot potato. And perhaps this is why the current view is that manager's need to steer, guide, facilitate these development efforts every step of the way. A reasonable approach given the turbulence that change can inflict on any software project. I speak more about managing developer efforts, change and other aspects of a software project later as its pretty interesting.
Benefits of any software system include aspects of user satisfaction and quality of service and other things over and above an economic benefit – which if isolated and the other things neglected can result in poor software performance perception. Sometimes as developers, we only see part of the vision for a piece of software we are commisioned to develop.
Software needs to be developed to relate to the environment and business that the software will work in – many aspects of inter-relation need to be considered when evaluating software’s benefit and potential. Very often only some of these are viewable in a developer’s conscious or perhaps they have been limited in exposure from higher levels of management. See my future prediction around the role of the independent "managing developer", aptly referred to as the MD because I think this role represents new owners of new companies...startups etc.
Complexity or Codeplexity(my word!) is another software is a problem. Complex systems are often part of disasters - space programs and industrial disasters, due to the exceeding the cognitive load to understand, plan and reason about them in varying situations. Maintenance is also a problem for complex systems.
I write software and I have been writing software for many years in many different disciplines and areas and I like to think that my passion for it is precisely because it changes but recently I've wondered do I know what it really is in the wider sense? It's it just a means to an end? This is a question that has developed in my mind this weekend.
Many aspects of society have realised or started to embrace that fact that computers and technology and by extension, software can serve a lot of uses and automate many things and as such, their influence in society is growing fast and approaching mainstream – if it is not already considered mainstream, particularly in developed countries.
Computers are changing the ways societies function, previous ways are now superceded by new ways that are facilitated by computers and thus old ways are no longer used. This forces people in society to deal with computers whether they want to or not – case in point, transferring money to bank accounts instead of having envelopes and people counting and dealing with them - another is the advent of Mondza for paying peers. Being at the forefront of this change, I have been apart of new inventions and seen old ones die away. But being in it, perhaps pervents me from seeing what its impact is on the world or the smaller environment I'm in -society. Have I not cared before? Why now? All interesting questions I ask myself from time to time.
Remind me of the ending of Men in Black,
where when you are in something, you forget that wider view is larger than your own.
Traditionally computers started to show economic cost reduction in automating manual jobs – this was and still is a benefit. There is, however, a negative view that it replaces people.
Computers as devices or pseudo-people are so flexible that they can be used to describe any problem and can attempt to deal with any problem – so are very useful in organisations, societies and individuals. They are like clay – however more so for organisations where lots of activities need to be accounted for and thus potentially a computerised solution exists for it.
Software, because of its ability to change things generally, provides new opportunities for organisations to use them, to devise new ways to make money etc.
Communication too is key in society and computers are pushing the boundaries of communications within them. Do we really appreciate how much change is happening around us?
Computers, through communication mediums such as email and the internet, have made them more widely used, not to mention recent developments through social networks among other things. I'll talk a little about how social media has affected me personally a bit later. Also, because communication is social and thus computers are becoming a greater part of society. Computers and communication go hand in hand. Computers are quite diverse in their application. It only made sense that social networking was embraced by society, which can be defined as communities of communicating individuals sharing stuff - ala internet.
in this information society, change is inevitable...
In having software which is classed as software systems, the question arises about how best to acquire any piece software in terms of getting at it to reap its benefits - particularly from a business investment point of view. Buy it, develop it or hire it? From my perspective, its all about developing it - am I software developer but from a business owner, it might be buying OTS (Off the shelf) software or other options(offshoring, outsourcing etc..) but the point is that bringing software into the enterprise needs to be considered and thought out. Its expensive, it can go wrong and it has repercussions but its also very useful when done right.
Another interesting insight is that software, computers and technology advances allow more opportunities. One such opportunity is perhaps automation, increases in accuracy, efficiency and cost-effectiveness in whatever they are designed to deliver/do.
A good example of how technology has changed with society, and as an extension me is internet banking and social media. Let me prise this idea apart...
Internet banking is the ability to interact with banking services which otherwise would require physical attendance at your bank. The impact it has on me has changed the way I can respond to money requirements like transferring money to friends and relatives easily and without seeing them or the bank.
It allows me to check my balance on my mobile phone, transfer pay bills and monitor my spending. It saves me allocating time to physically attend a bank and take time out of my schedule to do so. Sometimes I don't even think about it, it just happens and I do it. This really has been the first time I've thought about how things have changed but its weird that I've not considered that change, I've just been a part of it - like in MIB. All this just seems so obvious.
This also enhances the bank's bottom line in many ways, it doesn’t have to have to hire as many bank staff in physical buildings, thus reduces building and service cost that physical banks need to consider. It also means that they can communicate with me quicker and faster through my mobile phone. Online banking allows society to expedite interacting with money and allows to get on with their lives. And I'm probably only touching the surface of how technology, change in specific, has wel,l changed life as we know it.
It's truly incredible.
Now society has bank accounts and instead of getting paid physically using physical currency, money is wired to their bank accounts instead. This is safer too. It also means that you need a bank account in society.
Life without online banking would seem to have been going backwards, and taking time to do banking would take up more time in one’s day than otherwise, it would. We would have to plan to ‘go to the bank’ and perhaps the bank would be a more social place?
Computer technology has changed me in other ways too. Spare a thought to those that have to be dragged through these kinds of changes that many of us implicitly embrace every day and take for granted as I'm discovering while writing this. But before I wrap up this discussion on software and how its changed us, I have another example of changes to society and me in it, whether I like it or not and because I said I would earlier: Social Media.
Social networking is basically a means of social communication and interacting between people through the internet and via these sites on the internet. It allows people to voice opinions, share information and generally be sociable.
For me, these sites enable me to have a more global voice and feel connected to a larger society other than my immediate physical geographical area.
Advertising, research and marketing have embraced social media as a way to reach into these global societies. People can also influence companies by sharing opinions about them and communicating directly with them to voice concerns, frustrations or appreciation.
A case in point is seeing many people I follow on Twitter or who follow me voice their concerns at service providers products etc. The other is how Donald Trump embraces social media and is using it to the best of his abilities. This is a changing world, its changing because of computing, technology and society is changing because of it.
Companies use them (social networking sites) to be more connected with this community. You can follow and see what people are saying and this gives you the freedom to choose what information you embrace into your cognition. With the widespread use of social networks, they also exhibit abuse as would any society, which has its fair share of negative members and connotations etc. This and other aspects are magnified by the global social reach that these societies have.
Also, Fashion, status, humour and countless other types of societal tendencies can be embraced and shared through social networks.
Interestingly, without social networks, I’d perhaps have to work a little harder to obtain the information I’d want to have(maybe a good thing) and perhaps I’d focus on the more immediate concerns instead of worrying about the others. Though, I’d also need to make the effort to find a platform that reaches the people I want to talk to or hear from.
For example, this blog, its reaches people from all over the world, through social media sites like Twitter because I tweet my posts so people can read them. This is a platform essentially.
Without social media, I would perhaps be a lot less bothered by the information I receive that perhaps I would otherwise not care about. I’d probably embrace the decline of social media because I’m inherently not motivated by societies(or I think) and would rather pursue more individual and more introverted endeavours without having to share my experiences with anyone else.
But on the flip side, then I’d lose a platform to share my ideas with others and expose my creativity etc. Social networking sites.
But then again, as A dangerous miracle suggests, you are a team player whether you like it or not.
Anyway enough about why software is so interesting in the wider context, here is an interesting reflection I've had about developing software in general:
I’ve been reading about software, in a wider sense of the word, not in terms of technical solutions but in terms of theoretical manifestation of it solving problems, it merely existing and what its impact has on me and the environment in which its placed. Big world thinking.
One thing that I’ve realised in my years of being a software developer is that building software is more than just about how it do it. In fact, I’d argue that my orientation has been skewed towards it. Less has it been positioned against the impact of what I do, who its for and what difference it makes for someone that I do it for. In short, there is a disconnect between seeing the requirements and understanding the stakeholder's needs and requirements while I’m coding.
To bring in a software system into a society like any organisation presents all the benefits and disadvantages previously mentioned. You need to evaluate and reason about what they will be and are so you can develop a system that provides them – you need requirements but this might seem obvious to a developer: this is what is has to do…
Advantages of software systems usually relate to users of the system or organisations that they system will be used in. It needs to deliver something useful to them. I am worried that I still focused on how it done, being the technical wizard…
Customer satisfaction, stakeholder perception is affected by software that I write but do I think about this enough and am I being exposed to it enough? I think not.
Anyway, these aspects can be calibrated or concerned about and perhaps they should. The biggest problem I think is that these aspects are not exposed or shared with developers. That or developers are not concerned with them? Either way, it’s a problem. Because no matter how technical apt I am, the software I write is only measured by criteria that ultimately lies with that which I’ve not been exposed to – or allowed myself to be exposed to – the benefit and requirements of the system for the owner commisioning the software I’m writing.
This might also be due to the traditional large-company mentality where the division of labour and management causes this information not to trickle down to the coalface. This is why I think moving forward, developers will become more free-lance than ever before as they realise that they need to be a part of the whole picture and thus will try to find ways to escape the traditional problem of not being unable to see the wider picture of the software they are developing. They will be less inclined to be managed as this is a filtering of the fundamental requirements that make software they write successfully instead of merely instructed my managers to develop something and are only asked with being concerned about how it will be constructed not why.
So, the rise of the managing developer will come I predict, an independent freelancer in the same way as one would not have a surgeon or a doctor work on a wound after being told they he must because it hurts and he must make it not hurt.
Digressing a bit, models like the kano system attempt to reason about customer satisfaction and perhaps this need to be considered before the" how" is investigated by a developer.
It's very important to reason about customer satisfaction as it one of the indicators of a useful software system and this is normally one of the advantages sought in developing a software system within an organisation or group.
This is one of the “Whys” to have a software system produced – "whys" form part of the requirements that satisfy the whys. Personally, I've viewed the ‘Hows’ of the software developer world with perhaps more regard and embrace the ‘whys’ less… I'm realising that this is a problem as both are needed equally to be a better software craftsman.
But have I even heard the words "Volare" even used in my day jobs? No.
The Volare system of requirements process references the kano system of customer satisfaction.
The kano system reveals that many features that satisfy customers are only spoken about when they are experiences of actual performance. Unspoken features are expected and without them, all things go to hell. As I study Software engineering from an academic perspective, I realise how much academic knowledge is not implemented in business of commercial environments. The question is, is that my fault or someone else's. Arguably it's mine and this needs to change.
Truth be told a lot of this might be because I love ‘how’ its done to the extent that I’m not interested in why it done. This is perhaps an unexpected truth. So moving forward I’m embracing the need to know why and this is the study of knowing and understanding the problems that need to be solved and the requirements that my software must have to deliver it – more than simply finding a way to implement it… Obvious isn't it?
I’ve been reading a book about focusing more on the “whys” and less on the “hows” that I’ve traditionally enjoyed. These are the real "requirements" of my software.
A lot of this speculation has come about from my past experience in writing software for really small companies then really big companies and then really bureaucratic organisationsm then those that are not... Its also stems from my general interest in software engineering. I’m sorta, kinda starting to realise things about the way things are being done vs how that academic studies purport things should be.
The book is “Mastering the requirements process”. You can read about it here: https://www.amazon.com/dp/B008TVLH0E
In the first chapter, 11 truths are revealed about my poor-little “whys” aka requirements:
here they are with my summary of important themes i gleaned from them :
Truth 1: Requirements are not really about requirements
It focuses on the understanding of the business problem, what it is and how it might be solved. Correctly identifying the real problem. It’s a lot like a discovery – discovering the problem to be solved.
This is a hard one for me but it's not just about writing an advanced copy constructor or using a stack/linked-list/queue or some special cache in your software – it's about why are we doing all this for? Sometimes I let others worry about this but should I?
Truth 2: Software must be valuable to the owner commissioning its development:
The owner is an important stakeholder who ultimately pays for it and to whom deems the software produced as successful/useful/beneficial or not – particularly to him/her. The owner is buying a benefit when paying for a software system. You must know what this benefit is. The cost of the commissioning a piece of software must be directly proportional to the value or benefit he is buying it for.
It's important to determine what the owner’s values are, not just the user’s values. Ultimately they are aligned but the end game is the owner’s perception of the value the software he bought delivers. So, an emphasis is to be made on the owner of the software system. So, understand the owner’s problem. Don’t forget the owner paying for it all, not specifically the users of the system.
Truth 3: You need to know what the need is of commisioning the software while developing. So, in practical terms, this means the developer needs a shared vision and idea that matches the owner’s idea vision – this is in order to build the same predicted vision/idea.
This is when I realised my prediction and the convergence of the developer and manager role - the managing developer, the rise of the freelancer. See above
Truth 4: Building software and solving a problem taken independently doesn’t constitute a successful piece of software – building should not necessarily come without identifying the problem. Sometimes I like writing software more for what I can do while writing it than why I’m writing it – this is the disconnect I’m talking about – I’m a technology lover.
As an aside: There are those that would argue that prototyping identifies problems and identifies ways to solve problems while constructing something – however, this does require knowing the overall aim, and this process could be argued as a means to discover hidden, micro-requirements that weren’t fully articulated initially. Again, that’s if you were even apart of the requirements process, to begin with. And this is the next topic what is the requirements process and more importantly how you do effectively share it with developers.
In a nutshell, software doesn’t solve a problem alone. Thinking, planning and aiming software towards solutions does.
Truth 5: Requirements don’t need to be written – they do most emphatically have to be known. This is so that they can be reasoned about. The same idea and view of the requirements need to be shared - how to make all relevant parties see the same need and seated outcome? This is an important concept where one shares the owner’s goal with the developer.
Another thing that I’ve realised is how often do we actually document decisions that happen from time to time. Decisions lie on the cusp of changes and thus are an early warning change detection mechanism. There is a lot to be said about this, but that will be for another time.
Truth 6: Customer’s don’t always give you the right answer.
Many cases being told how to solve the problem doesn’t actually solve the problem because there is more to consider. A common problem is customers see an existing solution, and then modify it in an incremental way – this limits innovation but could have its merits if pragmatic, all things considered. People try to solve the problem too quickly and people accept solutions too readily. Lots to be said about this too.
Sometimes a requirement gathers aka a business analyst needs to persuade the customer that the customer's view of the solution is not wholly correct – they are missing some important parts of the bigger picture, which they have not considered.
This also raises the question, where is this business analyst? A far more sinister realisation is that there isn’t one and that he is, in fact, someone else! Are managers silently being business analysts - should they be? Lots to think about with this one too. Perhaps this is a pragmatic way to an economic problem?
Truth 7: Delivering requirements don’t come about by chance – they need an orderly process for developing them.
Any important endeavour needs an orderly process
Diverging from an orderly process stresses me out for exactly this reason. Things start to fly off the aircraft and things go wrong. Slow and steady with the right goal in mind, wins or finishes the race. There is a reason it called software engineering – it’s an orderly process of engineering, not fly by the seat of your pants.
Truth 8: Process doesn’t matter if you don’t understand the problem that needs to be solved or things that influence the problem that needs to be solved. Or indeed the solution. You need to understand the problem that all this work and effort is supposed to be directed towards.
What could be worse than the antithesis of 7 and 8? No orderly process and no understanding of what is the real problem.
Truth 9: There is no silver bullet.
If you are not prepared to think, nothing anywhere will help you. Sometimes you should think, always you should think, you should stop listening and doing and perhaps start thinking more. People aren’t paid to think for you and just hand you the problem to engineer. Sometimes it feels like this.
Tools don’t remind you what’s important, you should already know what is important without the aid of accessories. It's not about Git, or Docker (how) its also about them why.
Truth 10: Requirements, if they are to implemented successfully, must be measurable and testable. I can’t tell you how often I’m not told of the measurable qualities that a requirement must meet. However, unfortunately, I can tell you that I never ask from them!
Now, some reminders about requirements(whys) so that we can measure them:
A non-functional requirement is about how well a functional requirement is delivered, it’s the grease in the socket. You should convert requirements into numbers so you can see if they are above or below the required threshold for that requirement / true or false, success for failure. I don’t do this nearly enough.
We often and must not forget that knowing what requirements are is only part-way there - execution of the delivery of meeting any requirement is only possible if it has the associated instantiation of its realisation - a comparable and quantifiable number AKA a measurement for a requirement
We need a measurement for a requirement.
Truth 11: You, the business analyst(or in my experience, management) will change the way the user thinks about the problem because you’ll show them things they never considered – which were out of their view when they talked to you about the problem.
Truth 12: Requirements can come about by change.
Ok, I invested this myself and I think its super true.
You need to help people understand the problem as soon as possible.
Generally, in a quest to define a requirement, there are two types of requirement.
Functional requirements make the functionality useful to the operator. They are things that the system must do. They are things the system must do in the context of the owner’s business.
Non-functional requirements are what make the requirements acceptable, they also are what make people want to buy it or use it or be a part of it. They are often the gloss that makes the product shiny. Shiny is desirable, shiny makes things move faster - faster is good.
It's Interesting how non-functional requirements can be seen as the means to deliver or realise functional requirements. The shiny(non-functional) makes it move(functional) easily!
The non-functional requirements influence the “ease” of it all
Constraints are global requirements.
You can imagine that there is a hierarchy of stake owners, the owner being one of the highest ones and a user of the system is lower down. Delivering/meeting a user’s requirements facilitates meeting the owner’s but we should never lose sight of his/her.
When it comes to being a software developer, bizarrely ignorance is bliss but perhaps at your own peril.