New-age Transactional Systems - Not Your Grandpa's OLTP
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
The content has been bookmarked!
There was an error bookmarking this content! Please retry.
Posted by Srini Penchikala on Jan 19, 2009
Software development is challenging and lot of fun, but there are several factors that interrupt the teams from succeeding in IT projects. These factors are usually not the tools or technologies but it is the people that affect the success of software development projects. Venkat Subramaniam talked about the facts and fallacies of everyday software development. He gave a keynote presentation at the recent CodeMash 2009 conference on what developers and project managers should watch out for to ensure the success of their projects.
Here are some of the fallacies in software development Venkat talked about in the keynote presentation:
More money and time will solve our problems:
Venkat said it's important to have clear "business objective" when working on the projects. He showed the statistics that 64 percent of the features implemented in a project are rarely or never used and the longer the duration of a project, the lower the probability of that project being successful. Software developers need to develop what he called the "capable and relevant software".
It's gotta be good because it is from this large vendor:
He questioned if we really needed EJB which (in some cases) made the development complicated and heavy-weight. The developers are partly to be blamed for this situation (by using the EJB's for pretty much all use cases). One of the reasons for the adoption of EJB was that the competition (not the need) led the way. The need should determine technology, not technology determining the need and it's not a good evolution for standardization to occur before innovation. Frameworks like Rails and Spring have shown us that innovation before standardization does lead to useful and relevant development frameworks driven by the community rather than the vendors.
We're off-shoring because it will save us money:
Venkat talked about the current off-shoring model and if it's really working or not. He compared the model when companies did everything here in US and we still didn't succeed. So he asked, why would you put oceans between designers and programmers and expect different results? Off-shoring model has mainly become a "Fail-for-Less" strategy. He suggested that project managers hire smart skilled developers, those who can learn fast and then give them the sharpest tools, tools with high essence and low ceremony. A small team of highly capable developers is better than large team of below average developers. He asserted that off-shoring is here to stay so we should take the advantage of real talent worldwide.
Dynamic Languages are not safe:
He talked about Java being strongly typed and you can still get ClassCastException if you are not careful when coding in Java, so are Ruby and Groovy languages. Strong tying by itself doesn't improve the code. Compiler is useful, but is often overrated because no compiler can absolutely verify the programmer's intent. He also recommended the developers to take the advantage of new dynamic languages.
Static typing is essential for code clarity:
Venkat said with discipline, the developers can write readable, understandable, and maintainable code in any language. He suggested the following practices to keep the code clean and maintainable.
He also discussed the importance of unit testing in software development life cycle. Unit testing is the software equivalent of exercising and that's even if it's so good, more programmers don't really use it. It helps improve the health of the code and if code is testable, its design is good. Venkat concluded the presentation stressing that it's important to express the "intent" in the software.
Srini Penchikala currently works as Security Architect and has 17 yrs of experience in software product management.
18 agile and lean practices for effective software development governance
SCM best practices for multiple processes, releases & distributed teams
In today’s hyper-competitive world, later may be too late to adopt Agile development and this Roadmap for Success will help you get started. Download "Agile Development: A Manager's Roadmap for Success" now!
Great points!
Unfortunate that we still keep ignoring these. Software development is completely people driven and people dependent. In spite of all the myriads of processes and methodologies the right human resources still continues to be the single most important factor to software project success. More than thirty years ago Dr. Brooks has mentioned about the best programmers as "10 times more productive" as the worst programmers. Over the years the divide is still pretty apparent. Yet we keep proposing "factory model" and other such so-called "high" productivity development methodologies!
> He talked about Java being strongly typed and you can still get ClassCastException if you are not careful when coding in Java
Java is statically typed but not strongly typed; if it were strongly typed (like languages such as ML or Haskell are) you wouldn't get ClassCastException or NullPointerException.
> He also recommended the developers to take advantage of new dynamic languages like Scala.
Scala is statically typed, not dynamic. It has stronger typing than Java, which is one of its main selling points. Recommending it in a discussion about typing contradicts 'Strong tying by itself doesn't improve the code.'
(I don't know whether the misunderstandings are yours or Subramaniam's)
Hi Pete, It was my bad on the Scala reference, I updated the post to remove the reference. Regarding strongly-typed languages, here are some links:
Strongly-typed programming language
Java Overview
"Using a static, strongly typed language like Java puts the strongest possible constraint on the type of object at declaration time" (Scripting on the Java platform).
"Using a static, strongly typed language like Java puts the strongest possible constraint on the type of object at declaration time"
I'm very sure e.g. the Haskell and ML folks would strongy disagree with this statement :-)
Java is statically typed but not strongly typed; if it were strongly typed (like languages such as ML or Haskell are) you wouldn't get ClassCastException or NullPointerException.
Java is a strongly typed language. But it has a kind of dynamic typing represented in casts and reflection and that is where you get ClassCastException. So I would rather say it is strongly typed but not completely statically typed.
@Pete please ignore my previous comment. I just agree with you.
I'm very sure e.g. the Haskell and ML folks would strongy disagree with this statement :-)
True. Haskell and ML (F#, OCaml) compilers will try to put the least possible of constrains on your types!
in f x =x , f will take type "a" which is anything...
I'm aware the term has a variety of meaning, but that quote from the scripting article would just get laughed at by anyone with any exposure to strongly typed languages.
The wikipedia article has the specific trait of strongly typed systems I was alluding to -
"strong typing" include[s]: ... Disallowing any kind of type conversion. Values of one type cannot be converted to another type, explicitly or implicitly.
A programming language where ClassCastException can exist is not strongly typed - or at least, not as strongly typed a language as it could be. Java certainly does not have the "strongest possible" type system.
The presence of ClassCastException indicates one of the weaknesses in the Java type system. NullPointerException another (that any reference type can be substituted by an object of other type - null - at any point in the progrom).
Rejecting strong typing because of weaknesses in Java is contradictory, like rejecting GUIs because of Windows 2.0.
These are all good points that, despite being reminded of time and again, seem to fade away from people's memory exactly when they are needed the most.
Throwing money and people at a project, especially when the project is already facing internal challenges is always a bad idea. Overall project size is almost invariably challenging good quality, effective management (at both individual and project level), good and well-managed requirements and expectations with customers and on-time delivery. Large amounts of funding lead to attempts to "boil the ocean" for features and/or capabilities not required, at least not immediately. The long duration implied by big-spend projects further dilutes the focus and momentum and "invites" attempts of scope-creep.
Solutions from large vendors are not necessarily bad. Many times there are good (although maybe not always visible to all) reasons for going with them, even when heavy-weight. The art and science for architects and managers is to balance effectively the pro's and con's in each particular instance (and vis-a-vis the enterprise environment), removing extraneous considerations -- however well intended.
As to off-shoring -- as in geographically dispersed teams and team members -- it is important not for the (perception of) cost reduction --disputed by some studies and diminishing over time anyway due to improving wages in "off-shore" countries -- but for the simple fact that good people and subject matter-experts are not always conveniently located in the same city or even country when they are needed. Our project management methodologies and frameworks must evolve to effectively harness these people's capabilities wherever they happen to be located.
John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.
Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.
Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.
Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).
Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
9 comments
Watch Thread Reply