New-age Transactional Systems - Not Your Grandpa's OLTP
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
The content has been bookmarked!
There was an error bookmarking this content! Please retry.
Posted by Deborah Hartmann Preuss on Apr 16, 2008
Transforming Software Delivery: An IBM Rational Case Study
Case Study: IBM's Agile Transformation
SCM best practices for multiple processes, releases & distributed teams
In today’s hyper-competitive world, later may be too late to adopt Agile development and this Roadmap for Success will help you get started. Download "Agile Development: A Manager's Roadmap for Success" now!
The branches and merges of a lump of code has EXACTLY the same shape as that of the PERT Chart for the Project working on that code. I thought that is what the article would explain given that the figure near the title looks a little like an unfinished PERT. Even if you add in daily integrations this maxim remains true. So the PM decides the structure. It is never arbitrary or up to the Coder.
Great article, it covers the most commom troubles in mostly development enviroment even using other than XP methodologies.
Whatever pattern used to code versioning requires communication and team members commitment. I've been watching many teams misusing and doing a big mess on version control due to their lack of organization, comunication and stuff.
* Simple
Outstanding job describing and visualizing the thought process. I'm particularly impressed with the simple explanation on how to think about your release mainline's relationship with multiple development code lines.
Branching the code in this manner makes perfect sense (something we've been looking at for many months) - If you are only making code changes - everything works... but (as in most cases) you have database changes as part of your development - how do you make/merge these? the DB changes need to follow the same branch strategy as these changes are per line of code. There doesn't appear to be an easy answer - and I for one would like to here any opinions on this..
Firstly I'd like to join the chorus by praising the clarity and the great visualization of the problem. In a shop of 80 developers we have been pushing to get to this model and have met resistance particularly with merges, especially daily merges. You can attribute that to a mix of inadequate tools, folklore, bad experiences with cvs merge (yes cvs is not that great but the amount of effort to migrate has not been worth yet)...
We went in underestimating the cultural aspect of changing something as central to software development as source change management and it cost us numerous discussions and a consensus that got us almost there but not quite.
I'm hoping that this type of polished presentation can help people in similar situation as ours to lower the amount of resistance to change.
One pretty convenient way is to keep the DB changes with the code changes, treat them exactly the same way and check them in together.
So when you make code changes that require corresponding DB changes, update your DB script and create an SQL migration script as well. The migration script contains SQL for migrating schema and data from the old version to the new version. Everything is checked in along with the code, just another part of the source tree.
If DB changes are frequent then have a schema update routine execute automatically upon system startup. That routine executes any new migration scripts.
If the data is critical or the migration complex then you can implement automated tests to ensure that the migration script works correctly, by having it work on a snapshot of the production database.
I don't know about your context but this strategy has worked pretty well for me.
I have similar working model before. But there are always problem when merging from work branch to Trunk. It depends on Team disipline. There are still problems that "It works on my branch" issue. Especially for my team with multiple region developers, it always takes time to find out who causes the problem.
So, finally, we have an additional trunk between working branch and trunk called staging for integration testing the "Release" version. And it takes too much effort to do that. Any good way to resolve it?
First, let me recommend reading Wikipedia on this: Revision Control and Comparision of revision control software.
The basics of this article works fine for the enviroments where it could be used -- when you have distributed systems, where the developers of each might be completely different software houses using different methodologies, it can't be used as is. But YMMV according to the software you are using.
This article seems to be predicated on centralized versioning systems with strong trunk support. That doesn't mean you need that -- in fact, I believe other types of versioning would work even better.
Subversion, the king of free revision control software, would work fine with it, though you'd use branching instead of trunks for some stuff, most likely.
Distributed versioning systems would probably differ a lot in the way of working, but not the very basics. So you might not have a "trunk", and instead have your local repo, and you just send the patch to the stable repository, but the ideas still apply.
The only thing I really, really disagree with is the check out in the morning/check in at night modus operandi. One change, one check in. If you stick to this rule, then merges and bug trackings will come much more easily.
So, for instance, don't check in a half-done refactoring. Yes, the size of the merge for the others on the next day might be smaller, and they might notice earlier that what they are using is gone, but they won't have the whole picture to go by, and neither will anyone looking back to see what was being done when a bug crept into the code.
Likewise, if you refactor and create a new functionality (two tasks) in the same day, do a check in for each task, not just one with everything. First, it's easier to roll back selectively. Second, it's easier to understand what was done for what reason.
When you explain the Firm<->Soft concept, there is R2.4.2 above R2.4 - which is kind of wrong. Since each bug-fix in R2.4 we would also want to be propagated into R2.4.2, and change is flowing down. Thus R2.4 should be above R2.4.2.</->
This article is very nice overall, but when confronted with a person saying that they want to use branch-per-release.... Just because he used to have that with ClearCase on his previous job. There are no concrete benefits that can be pointed out. Since each of these concepts applies just as well to a model of branch-per-release. So it's quite frustrating.
I don't agree. The relationship between 2.4.2 and 2.4 is like 2.4 to Main. Thus normally bug fixes go to 2.4.2 (if it is being maintained) and then flow to 2.4 (and then to Main).
If you make the bug fix on 2.4 then chances are you would branch 2.4.3 subsequently.
This is a very nicely done article.
I would prefer to see a reference to Laura's name in the text of the "Big Picture" section (as well as references). She deserves higher billing :)
The URL for my article you linked to is slightly wrong. It should be:
Branching and merging - an agile perspective
I found the article well written, but is there a balance between simplifying and over simplifying at the expense of productivity and how much can be accomplished in a given sprint? More sophisticated version control tools like Accurev seem more suited for agile development environments than what you describe here using Perforce, which seems to me like a trying to fit a round peg in a square hole. You've just given it a bigger hammer to drive it home, but it seems like a whole lot of hurt for nothing.
I tried to find a comparison outside of the vendor sites, but this seemed reasonable.
www.accurev.com/scm-comparisons/accurev-perforc...
This looked like a good resource to explain in more detail what I mean.
www.accurev.com/agile-software-development.html
MergeMagician (www.mergemagician.com) is focused on automated merging and addresses many issues raised in this article.
FYI russian translation is here: anton-nix.livejournal.com/2865.html
There is a great post by Vincent Driessen "A successful Git branching model" (nvie.com/git-model) which presents a GIT branching pattern which is very similar. He describes all necessary git commands in detail.
Furthermore, he developed a git extension "gitflow" (github.com/nvie/gitflow) which provides high-level repository operations for this branching model.
Hence it is very simple to start implementing this model.
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.
Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.
Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).
Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
18 comments
Watch Thread Reply