New-age Transactional Systems - Not Your Grandpa's OLTP
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
The content has been bookmarked!
There was an error bookmarking this content! Please retry.
Posted by Vikas Hazrati on Oct 27, 2009
Keith Swenson, recently compiled a list of 26 hints for Agile software development. Keith suggested that he frequently collects nuggets of wisdom on various topics and the list is a distilled set of hints which really matter for Agile software development.
A comment on his post suggested that most of these hints may not be Agile specific and would be true for better software development and design. Keith responded that, for seasoned Agile practitioners many of these hints might sound routine but there is a larger audience out there who are agnostic to these practices. He added,
I am working with some teams in Japan that use a very strict waterfall model for development. To that team, about half of these are “surprising” and possibly even radical pieces of advice. Things like “write the test before the code” and “never implement before needed” are radical concepts to them. They pride themselves on “completeness” of implementation, to the point of inventing use cases which are not customer driven. The result is overbuilt code, another kind of waste. They wait 6 months sometimes to implement the tests. To those implementing in strict waterfall, the test is a “crutch” that should not be needed by developers doing their job correct. Surprising?
Some of the 'not so common', interesting hints suggested by Keith were,
Read more through the complete list of hints and leave a note if you feel that an important one is missing.
Transforming Software Delivery: An IBM Rational Case Study
Agility at scale, become as agile as you can be
In today’s hyper-competitive world, later may be too late to adopt Agile development and this Roadmap for Success will help you get started. Download "Agile Development: A Manager's Roadmap for Success" now!
I'm not very sure about "Premature optimization is the root of all evil.", can somebody explain this?
Is this something similar with Lean principle - Systemic Optimization?
I guess a lot of time is spent in trying to optimize code etc which actually might not be an issue when the system is functioning. This is seen a lot when endless time is spent on optimizing the method and once into production you realise that main cost of the functionality is spent in interfacing with external systems like databases, web services, legacy systems etc etc and excessive code optmization upfront may not be of any help.
I'm not sure this always applies although in most cases is probably good advice. One counterexample I think applies is where the project functionality crosses tiers. For example, suppose my project requires some number of stored procedures or SQL database coding in conjunction with a .Net web app. It might make sense to go ahead and work on the database code along with the app in conjunction.
This is actually advice from Donald Knuth originating on the Art of Computer Programming. It means that you need to complete your program before you optimise it. Humans are very bad at guessing where a computer spends most of the time, so if you prematurely optimise, you will just waste time.
See en.wikipedia.org/wiki/Program_optimization#When...
The key is the word "premature". It implies that you've started to 'optimize' before having a concrete requirement to optimize towards. Optimization should always move you closer to some required goal.
The "root of all evil" part is in regards to the lack of awareness of the project's true requirements. What happens when you are lose awareness/consciousness of the project's true requirements? You spend all your time shaving yaks [1] instead of doing something truly useful (to your customers).
~~~
[1] The Productive Programmer, Chapter 4. Automation > Don't Shave Yaks, Pg. 67:
my.safaribooksonline.com/9780596519780/I_sect13...
As others already said it's about optimizing before you actually know what needs to be optimized and Don Knuth (www-cs-faculty.stanford.edu/~knuth/) said it.
You should always gather some data by utilizing profiling, tracing, code overage tools and the like to find the hot spots in your code which need to be optimized and after optimizing it measure again to see what changed.
Unfortunately many developers just "think they feel" the right place to optimize and spend too much time optimizing the wrong thing.
Regards,
Robert
--
Robert Berger
Embedded Software Specialist
Reliable Embedded Systems
Consulting Training Engineering
Tel.: (+30) 697 593 3428
Fax.:(+30) 210 684 7881
URL: www.reliableembeddedsystems.com
Getting case #1 working before starting case #2 doesn't mean that you're supposed to finish working on one component of the system before the other. It means getting a business case done, before moving on to the next.
In fact, in the example you stated, it is a good idea to build the .NET app and the database in step, in order to fulfill the 1st business case before moving on to writing the .NET code, and the stored procedure required for the next case.
I think it's not just that you're optimizing the wrong spot, that's wasteful - but I think it points at a bigger problem.
When you optimize a program, you almost always limit it's flexibility, and make major tradeoffs in readability and generality.
When you do those things prematurely, you're making your code far less adaptable (read: more expensive) to change. And having things be easy to change is what agile is all about. That's one of the reasons we tend to write so many tests - make things resilient to change.
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.
Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.
Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).
Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
8 comments
Watch Thread Reply