"Systems Development": a New Discipline for a New Education
Do university programs adequately prepare software developers for today’s world of work? Now that Agile software development is being adopted at an unprecedented rate, developer skillsets particularly need broadening. In this article Dr. Dave West discusses a nascent discipline, “Systems Development”, which emphasizes humanity, craft, design, creativity, innovation, and emergence - in contrast with current university computing disciplines, infused with a value system centered on formalism, control, engineering, and determinism. In his view, Systems Development should be called “Reality Construction” i because it is based on the fundamental premise that everything a Systems Developer does is necessarily integrated with, and necessarily changes (reconfigures), an existing complex system - the human, organizational, socio-political, and cultural system we call reality. West speaks from experience when he proposes a better educational experience that replaces the sterility of today’s classrooms and labs with the workshop, or “bottega” – the approach he is currently using in his program at College of Santa Fe.
Our profession remains broken.
Yes, great strides have been made to improve process, quality, and professionalism and, in some cases we are able to deliver fantastically complicated reliable and usable software. Nevertheless, as an industry we still struggle to achieve a 30-40 percent “success” rate and an even lower “success within time and budget” rate.
Numerous factors contribute to the general malaise: false metaphors (engineering); ambition (we already solved the easy problems); pace of change (technology and the world); and blurring of boundaries between technical and human systems.
A major contributor is the widening gap between what we learn in school (and how we learn it) and what is required of our professional selves.
It is almost certainly past time that we defined a discipline that better fits what we actually do and create an alternative mechanism for educating individuals in that discipline.
Limning the Territory
The ACM-AIS-IEEE Computing Curricula 2005 Overview Report ii outlines a complicated overlapping set of related disciplines that include: Electrical Engineering, Computer Engineering, Computer Science, Software Engineering, Information Technology, and Information Systems.
Of the six, the first three (EE, CE, and CS) focus on the machine and the theory behind the machine. (Part of CS concerns itself with the virtual machine via operating systems and language compilers).
Software Engineering and Information Systems approximate our profession while Information Technology focuses on the infrastructure especially networks, security, and administration.
Nascent in the ACM document is the beginning of a definition of another discipline – Systems Development (SD).
Systems Development incorporates skill sets from several of the ACM classifications: programming from computer science; software, process, and tools from software engineering; and an applications focus from information systems.
To this mixture, SD adds: an extended knowledge of systems (General, composite and complex), a deeper understanding of the history and philosophy of computing and science, significant understanding of business and management, and multiple topics in philosophy, mathematics, anthropology, art, humanities and more.
All six of the ACM computing disciplines are infused with a value system centered on formalism, control, engineering, and determinism.
In contrast, SD emphasizes humanity, craft, design, creativity, innovation, and emergence.
Systems Development should be called “Reality Construction”iv because it is based on the fundamental premise that everything a Systems Developer does is necessarily integrated with, and necessarily changes (reconfigures), an existing complex system - the human, organizational, socio-political, and cultural system we call reality.
SD delivers altered states of reality.
The ideal architect should be a man of letters, a skillful draftsman, a mathematician, familiar with historical studies, a diligent student of philosophy, acquainted with music; not ignorant of medicine, learned in the responses of jurisconsults, familiar with astronomy, and astronomical calculations
Vitruvius 25 B.C.
Vitruvius’ architect had to be a polymath – had to have a significant degree of competence in all fields of extant knowledge with significant skills in specific areas.
The same is true of a professional (Master) Systems Developer. (With the caveat that no one, today, can be conversant with ALL extant knowledge.)
The path to mastery passes through several stages:
- Rote understanding of concepts and vocabulary.
- Applying the knowledge under the direction of others.
- Applying the knowledge when working alone.
- Applying the knowledge in a novel context.
- Ability to mentor others.
- Ability to teach others (groups as well as one-on-one).
- Ability to make an original contribution or extension to a given area of knowledge.
A professional should be able to exhibit stage 4 abilities for the entire body of knowledge and stages 5-7 for a significant subset.
Excepting stage one, mastery is always based on one’s ability to actively do something, not simply and passively know something. This bias is an echo of Giambatisa Vico (1668-1744) who asserted, “We are sure only of that which we build.”
All Extant Knowledge
The two previous sections suggested that the domain of knowledge, and the degree of mastery, for System Developers is rather expansive.
While that knowledge domain is not literally, “all extant knowledge,” it is significantly greater – at least double – the breadth of knowledge expected of university graduates with a Masters degree in any of the ACM curricular categories.
I cannot detail the contents of the knowledge domain in this space. [In part because it is still being defined and refined in preparation for opening the first SD Bottega in late spring 2009.] Instead I would like to move the discussion to modularization and learning environment.
It would be impossible to deliver the scope of content envisaged for Systems Developers if it were modularized in accordance with the typical credit hour / course / semester / academic year tradition. Doubling content would mean doubling the time required to complete a degree from 6 years to 12 – something that would never happen.
Instead of credit-hours and courses, we propose dividing the knowledge domain into much finer grained discrete knowledge units. When we did this at New Mexico Highlands (see below) we used the default term competency to represent a knowledge unit.
The metaphor of a mosaic is useful to illustrate knowledge units (competencies) and the relationships among them. The mosaic represents the entire domain of expected knowledge – each discrete tile represents an individual competency. It is important to note that each “tile” represents more than knowledge – it also includes aspects of practice (application of the knowledge), skill, and experience.
As is true with any picture, the mosaic exhibits themes – patterned relationships among subsets of the tiles comprising the mosaic.
Each tile has an associated color, indicating the discipline or field most associated with the knowledge represented by that tile. Units of philosophy, anthropology, literature, mathematics, history, science, etc. - are distributed across the entire mosaic, showing at-a-glance the relationships of knowledge across disciplines and achieving a degree of interdisciplinary integration impossible in a course-based program.
The mosaic provides an immediate gestalt view of the entire domain to be mastered. Themes v provide focus without sacrificing relationships to elements outside of that focus. Colored tiles show contributions from all disciplines – in context, with cross-disciplinary relationships obvious. Students not only know that they have to know Z, they know why, and they know how to apply Z as they learn it.
How big is this mosaic – how many tiles?
At New Mexico Highlands (experience report below) we defined a little over 500 competencies for the SD “major.” That number did not include liberal arts and other required courses.
A reasonable estimate, based on both the Highlands experience and ongoing efforts to define the set of competencies for SD, yields a mosaic with 1,000 – 1,500 tiles.
Although few would deny the possibility of that many tiles (as educators we are constantly battling the need to add more subject matter to our curriculum); most would raise the purely pragmatic objection – you cannot put that much breadth and depth into a curriculum and expect anyone to master it in the 6 years typically allotted to completing a Bachelors plus Masters degree program.
This objection, however, is not grounded in any understanding of how much or how fast individuals can learn. Instead, it is based on assumptions of how much learning you can pack into a 16-week course that meets for 3 50-minute periods per week.
How fast individuals and teams can learn a given body of knowledge is not unlike the problem of how fast an individual or team can complete a software project. Agile development suggests the mosaic (a collection of tiles) be considered akin to a ‘project backlog.’ Given a time-box of arbitrary duration, each student selects a set of competencies from the product backlog to mastered in that time interval – a ‘sprint backlog.’ At Highlands we call the sprint backlogs, individual learning plans (ILP) and used a time-box interval of 6 weeks.
Because all but the first stage of mastery requires ‘doing,’ it is essential to tie the ILP to actual project work, preferably real-world project work.
Each student determines their individual “learning velocity” relatively quickly. It is then a simple matter to plot how long it will take to complete their education.
Learning velocity is not simply a matter of individual difference. The proper environment can make a huge difference.
Bottega (the classroom)
Imagine replacing the sterility of today’s classrooms and labs with the workshop – a bottega vi - where Leonardo worked and studied.
- A “storefront” where goods and services are produced and delivered to paying customers.
- A workshop simultaneously engaged in the craft, in building the tools and discovering the techniques that advance and support the craft, and teaching that craft to apprentices.
- A place noisy with multiple projects and activities; walls and benches covered with works in progress.
- A place filled with the tools of the craft.
- With room for lounging, and eating facilities as well.
- An intellectual center that is a “must visit” for masters, scientists, and thinkers visiting the area. A center overseen by local masters and journeymen.
- A fountain of innovation and creativity!
- An environment and atmosphere that is very self-consciously multi- and inter-disciplinary; that mixes theory and practice almost without differentiation.
- A place full of music, conversation, laughter, and even loud argument.
- A place to share food and drink, perhaps sleep, and even love vii.
A bottega consciously mimics the ideal workspace advocated for agile software development: open, lots of whiteboards and communication artifacts, movement, collaboration, and socialization.
To be maximally effective, students need to spend large blocks of time in the bottega; ideally most their time.
The bottega operates as a “one room schoolhouse” and everyone, regardless of degree of mastery, is in the same place and exposed to everything in that environment. If something occurring in one part of the room sounds interesting or relevant, the student is free to roll their chair across the floor and participate.
In this kind of environment a lot of knowledge is absorbed non-consciously, almost by osmosis. When the time comes that an individual needs to consciously acquire and apply a bit of knowledge, that process is accelerated by virtue of the reservoir of non-conscious “background learning” they have already completed.
Personal responsibility, deadlines, and understanding and satisfying customers are all aspects of being a master. Commercial product development provides the educational and experiential opportunity necessary for students to achieve those aspects of mastery.
Can this idealized environment be created, maintained, and can it work?
Experience Report – New Mexico Highlands University
In August, 2004, 34 students, freshmen through graduate students, entered a unique program in Software Development at New Mexico Highlands University.
The degree program had no courses. Students were expected to master 500+ competencies in their “major” plus the usual general and liberal arts course requirements in order to graduate with a Bachelor of Arts degree.
The program was apprenticeship-based: everyone in the program was able to earn money working on real-world software delivered to paying customers with real-world timelines.
Fifty percent of the students were female (extraordinary for a computing degree) and 60+ percent were minority, and almost 80% were economically disadvantaged.
Two full-time faculty, Pam Rostal and Dave West, designed and operated the program. Internationally known master developers (e.g. Linda Rising and Ron Jeffries) visited the program, working with students (pair programming) and contributing their expertise via hands-on mentoring and conducting other types of learning sessions.
All development work and instruction took place in a “one-room-schoolhouse.”
Our studio environment included: island work areas, wall-to-wall whiteboards, hard floors and wheeled chairs. Couches, a refrigerator, and a microwave provided some nominal but much used amenities.
Significant time was spent fostering “culture and community.” Ceremonies were used to initiate people into the community and advancements in apprenticeship levels. Pot-luck meals (sometimes catered meals) were incorporated into our iteration retrospectives and ceremonies. Colored shirts denoting apprenticeship level, and the ubiquitous presence of the program’s logo, assisted in creating a sense of identity and belonging.
X bar times 10 alluding to the goal of graduating professionals ten times better than average. Software is our medium, Craft (Mastery) is our goal, People are our focus, Systems is our perspective, and Agility is our process.
Our freshman retention rate was 100% between first and second semesters at a University that averaged less than 50% for the same interval. Forty additional students – mostly freshmen – had committed to enter the program in the fall of 2005.
Seven students in the inaugural class were offered full-time employment by the end of the first year, 2 were accepted to graduate school, and one was able to concurrently fulfill requirements for a Masters degree in computer science.
Eight students co-authored papers and delivered presentations at refereed conferences, OOPSLA in Los Angeles and Agile in Denver.
Lessons learned from the first (and only viii) year the program was in place, include:
- Students learned far more, in shorter time. Based on this experience we believe that the average entering freshman will be able to complete the entire SD curriculum (twice as much learning as the average 6 year education) in less than five years.
- The depth of learning and understanding exceeded expectations.
- Students rapidly evolved a process of self-directed learning.
- Students selected, installed, and configured their own tools.
- The “strong sense of community,” “human values (software by humans for humans),” and the “open collaborative learning environment” were the factors women students consistently mentioned as the primary reasons for entering and staying with the program.
- Student’s learned teamwork and project management skills concurrently with technical skills.
- Students managed all aspects of all projects, not faculty.
- Computer Science faculty that had opposed the new program became active participants the second semester.
- Students averaged 30 hours a week in the studio.
- The “noisy” environment was not a distraction – it was an asset.
- The quality of student work was comparable to professional work – according to our real-world paying customers.
- Even the most junior apprentices made valuable contributions to their projects. They did not inhibit or slow down the work in any way.
- Both Pam and Dave averaged 70+ hours a week in the studio. (This time commitment would be unsustainable in the long run.)
When the program was cancelled, students organized a letter writing campaign, arranged a meeting with New Mexico Governor Richardson’s staff, and solicited endorsements from some of the nationally prominent individuals they had met at conferences. This was to no avail, but it reinforced the conviction that the program should continue, and efforts were initiated to replicate the effort at another college.
A more detailed report on West’s NMHU program was published on InfoQ in 2006: Experience Report: Agile Development Apprenticeship at NMHU.
Moving Forward – Transcendence Corporation, The College of Santa Fe, and the Santa Fe Complex
Transcendence Corporation was formed during the Highlands experience – to be the contracting agent for the apprenticeship program. (The University was forbidden, by state law, from entering into that kind of work contract.)
The College of Santa Fe is a small (less than 2,000 students) liberal arts college with very strong programs in the fine and performing arts.
The Santa Fe Complex offers a bottega environment, expertise in math, science, visualization, agent-based modeling, applied complexity, and the “intersection of art and science.”
A first approximation of the program described above was established at the College of Santa Fe in the fall, 2007.
In the fall of 2008 the existing program will be redefined as a five-year competency-based Master of Science program.
During the fall 2008 and spring 2009, the three organizations will work cooperatively to:
- Convene a workshop of interested educators and professionals to finalize the full “curriculum mosaic” discussed above.
- Offer bottega style targeted workshops for students at the College and advanced placement high school students.
- Recruit an inaugural cohort of 50 students including at least 10-15 high school juniors and seniors and 10 “transfers” from the College.
- Solicit projects (we offer off-shore rates and, based on the Highlands experience, higher quality); internships, and externships.
- Establish sponsorship relations with organizations – especially national and international consulting companies – that cannot recruit suitable graduates and have been forced to create their own in-house boot camps and training programs.
- Open the first bottega in late spring / early summer of 2009.
Anyone interested in working with us in this effort, (especially the curriculum workshop, contracting for project work, or sponsorship relations), is expressly invited to contact the author.
About the Author
Dave West has been a professional software developer for too many years to count, in roles ranging from programmer to Director of IT. For the past fifteen years he has split his time between consulting and academia. Currently Professor of Business, Innovation, and Technology at the College of Santa Fe: formerly at New Mexico Highlands University and the University of St. Thomas. His own education was an eclectic mix that included Asian Philosophy (undergraduate at Macalester College), Computer Science and Cultural Anthropology (graduate degrees at University of Wisconsin-Madison) and a Ph.D. in Cognitive Anthropology (also at UW-M). He is the author of Object Thinking (MS Press Professional, 2002) and is nearing completion of a second book on “reinventing software development.” Any expertise he possesses is focused in the areas of objects, agility, design, innovation, and change.
Notes and References
i I first encountered this phrase and idea in Christiane Floyd, et. al.’s book, Software Development as Reality Construction. Springer-Verlag. 1992.
ii Computing Curricula 2005 – The Overview Report. Copyright © 2006 by ACM and IEEE.
iv I first encountered this phrase and idea in Christiane Floyd, et. al.’s book, <Software Development as Reality Construction. Springer-Verlag. 1992.
v I see this mosaic in terms of abstract rather than representational art – a theme would be a bounded subset of tiles that delineated a focus without eliminating the connection to the whole.
vi The core of my description – which has appeared in numerous publications - of a bottega comes from Fritjof Capra’s wonderful book, The Science of Leonardo. Doubleday. 2007.
vii The Greeks considered Paideia to be carried out by the aristocratic class, who were said to have intellectualized their culture and their ideas; the culture and the youth are then "moulded" to the ideal. Starting in archaic times, love played an important part in this process as adult aristocrats in most cities were encouraged to fall in love with the youths they mentored – Wikipedia.
viii It is essential to talk briefly about why the program was in place for a single year – to make it perfectly clear that termination had nothing to do with the program itself. It is not a pretty story, and it would be hard to adequately describe the perfidy of the individual responsible. Two months after the program was approved and budgeted by the Board of Regents, that same board hired a politician with no educational background or expertise to be President. According to Board minutes - the primary objective for this President was to change the ethnic composition of the faculty. To do so, the President did everything possible to pressure tenured “carpet bagger” faculty to leave, unjustly denied tenure to faculty, illegally hired administrators, and much more. His actions earned sanction of the University by the AAUP, a multitude of lost lawsuits costing the University millions. He was finally fired and is currently under indictment for allegedly receiving kickbacks from a federal construction project while a state senator. The faculty leading the new program were on the President’s hit list. When it became obvious that his ongoing efforts to destroy the program were failing, he ordered it terminated – without consultation, advice or consent of the faculty, Board of Regents, or students.
Jose Luis Rodriguez
Looking forward to see the results of the first bottega, It would be more motivational if the results can be statistically measured in terms of "traditional" thinking so we can compare LOC performance, defect injection density, costs (per project and per career) and laboral satisfaction.
Still, the 10x is arguable...
Brazil has alrealdy adopted this paradigm
I am professor of Computer Systems and Information Systems programs since 1996.
Since then we are adopting a curricula that includes classes like "Real World Software Development". That is a course on the 3rd year were the students are organized in teams and invited to present solutions for problems presented or identified in our professional community. The work is fashioned so the students must deal with issues related to project management, software analysis and design, requirements engineering, relationship, communication and marketing.
Another interesting approach is the "Academic Software Contest", which is a result of the projects presented by students of different universities. The projects are evaluated by a board of examiners composed by IT and academic market professionals. Frequently are used different software development languages, different project management methodologies and so on. It is not rare that many of the works presented became market solutions and give birth to the students first business.
For more information see (in Brazilian Portuguese):
Missing real world experience with academic training IS essential. Trying to include overall management and broad process understanding is, I'll suggest, misguided.
Teach production through both acadenics and productive experience, in whatever process context works for the production shop. By all means, explain that process philosophy.
Then let them work for a living, in multiple production shops that are NOT yours. Learn real-world dvelopment process in multiple contexts. With the bacjground you propose, they'll readily discern conceptual similarities and limitations of different processes.
Then they have the "bag of tricks" available for use. No single development process works in all contexts and certainly not with all developers. Just as "Management" must be adapted to the organizational context and the staff being managed, so systems development should be fitted to the project and the team.
I fear you'll confuse your students by trying to do too much.
Shane Hastie Aug 23, 2014
Srini Penchikala Aug 21, 2014