Day Two of the 9th annual QCon New York conference was held on June 14th, 2023, at the New York Marriott at the Brooklyn Bridge in Brooklyn, New York. This three-day event is organized by C4Media, a software media company focused on unbiased content and information in the enterprise development community and creators of InfoQ and QCon. It included a keynote address by Alicia Dwyer Cianciolo and presentations from these four tracks:
- Designing Modern Reliable Architectures
- Hosted by Silvia Esparrachiari, software engineer at Google.
- Provides a diverse collection of reliability strategies covering finance applications, gaming platforms and other real-world segments of the tech industry.
- MLOps - Production & Delivery for ML Platforms
- Hosted by Bozhao (Bo) Yu, founder of BentoML.ai.
- Explores the latest trends, best practices, and case studies in MLOps, and discusses how they can be applied to improve the reliability, scalability, and efficiency of machine learning systems.
- Next Gen Fintech: Performance, Complexity & Privacy
- Hosted by Neha Sardana, vice president and senior developer at Morgan Stanley.
- Provides solutions to the challenges of data breaches, privacy, security, and performance, and stays on top of the latest trends and best practices in Fintech.
- Optimizing Teams for Fast Flow - Surviving in the Post-Agile Aftermath
- Hosted by Katharine Chajka, senior product manager at Planview Flow Methodology.
- Focuses on understanding how to identify the specific issues slowing down the flow and finding changes that resolve those impediments to make the most of the time and money spent on change.
There was also one sponsored solutions track.
Danny Latimer, content product manager at C4Media, kicked off the day two activities by welcoming the attendees and introduced Daniel Bryant, InfoQ news manager, who discussed InfoQ news activities and the InfoQ core values: information robin hoods; best, not (necessarily) first; facilitators, not leaders; and content that can be trusted. Pia von Beren, project manager & diversity lead at C4Media, discussed the QCon Alumni Program and the benefits of having attended multiple QCon conferences. The aforementioned track leads for Day Two introduced themselves and described the presentations in their respective tracks.
Keynote Address: NASA'S Return to the Moon
Alicia Dwyer Cianciolo, senior technical lead for Advanced Entry, Descent and Landing Vehicle Technology Development at the NASA Langley Research Center, presented a keynote entitled, NASA'S Return to the Moon: Managing Complexity in the Artemis Program. Cianciolo started her presentation with the famous quote from John F. Kennedy's speech at Rice University in 1962: "We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard." With that in mind, she introduced the Artemis program, considered the "sister" to the Apollo program, as a collection of projects, namely:
Artemis = Space Launch System + Orion Spacecraft + Human Landing System (HLS) + Extravehicular Activity and Human Surface Mobility Program (EHP) + Gateway
Cianciolo currently works on the Human Landing System. Artemis and its component projects were designed as a collaboration for space missions. After introducing each of these projects, she provided background and orbit information on previous and upcoming Artemis launches.
Artemis I, launched on November 16, 2022, and splashed down on December 11, 2022, featured the Space Launch System and Orion projects. Artemis II, scheduled to launch at the end of 2024, will feature the Space Launch System and Orion projects and include a flight crew: Reid Wiseman (commander), Victor Glover (pilot), Christina Hammock Koch (mission specialist) and Jeremy Hansen (mission specialist). The plan for Artemis III is to land on the moon and will feature the Space Launch System, Orion, HLS and EHP projects and include another flight crew that is still to be determined.
She described the complex operations related to the HLS that includes mission segments, contracts, landing requirements and gateway orbit. Apollo 11 through Apollo 17 landed on or near the moon's equator. The landing plan for Artemis III is to land within 6° latitude and less than 10° surface slopes on the moon's south pole due to the rough terrain. Another challenge is the amount of daylight that often changes. The crew will need a six-day window of daylight and to be in constant communication with Earth. "What could go wrong?" Cianciolo asked.
The hardest part of going to the moon is talking to people, Cianciolo said, as she recalled her experience in which it took six months to resolve this seemingly simple issue related to sharing a bathroom space.
The plan for Artemis IV is to land on the International Habitation Module which will feature the Space Launch System, Orion, HLS, EHP and Gateway projects and include a flight crew to be determined.
Highlighted Presentations: Performance, data structures, and ChatGPT
Maximizing Performance and Efficiency in Financial Trading Systems through Vertical Scalability and Effective Testing was presented by Peter Lawrey, CEO at Chronical Software. Lawrey kicked off his presentation by discussing how allocating objects may have an overhead of 80x in comparison with garbage collecting them. He maintained that the loss of objects can be allocated, but recommends against this strategy that has been a practice. As the legendary Grace Hopper once said: "The most dangerous phrase in the English language is: 'We've always done it that way.'"
He provided various analyses and benchmarks on why allocations don't scale well. Allocations can spend approximately 0.3% of the time in the garbage collector. The concept of accidental complexity is complexity that is not inherent in a problem, but rather in a solution that can be removed or reduced with a better design or technology choice.
Lawrey provided many examples and analyses of accidental complexity that included memory usage analysis from Chronical Queue where most of the allocation activity was from the JDK Flight Recorder. In many applications, especially in the financial industry, selecting a "source of truth" can significantly impact the latency and complexity of the application.
The concept of durability guarantees identifies critical paths for performance, but are often considered the largest bottlenecks. Examples include: a database; durable messaging that is guaranteed to be on disk; redundant messaging; and persisted messaging that will eventually be on disk. Lawrey introduced Little's Law, a founding principle in queueing theory, as L = λ, such that:
- L = average number of items in a system
- λ = average arrival rate = exit rate = throughput
- W = average wait time in a system for an item (duration inside)
Little's Law is applied in many aspects of system design and performance enhancement. The higher the latency, the higher the inherent parallelism required to achieve the desired throughput. On the opposite end of that spectrum, the lower the latency, the inherent required parallelism is minimized. Traditional object allocation in Java can impede performance, especially in high throughput scenarios, creating a bottleneck that hinders vertical scalability. By minimizing accidental complexity and using an event-driven architecture, vertical scalability can be achieved.
Donald Raab, managing director and distinguished engineer at BNY Mellon, and Rustam Mehmandarov, chief engineer at Computas AS presented Performance and Scale - Domain-Oriented Objects vs Tabular Data Structures. Raab and Mehmandarov started their presentation with a retrospective into the problems with in-memory Java architectures using both 32-bit and 64-bit memory circa 2004. In the 32-bit world, it was challenging for developers to place, say, 6GB of data, into 4GB of memory. The solution was to build their own "small size" Java collections.
The emergence of 64-bit provided some relief, but total heap size became an issue. Compressed Ordinary Object Pointers (OOPS), available with the release of Java 6 in late 2006, allowed developers to create 32-bit references (4 bytes) in 64-bit heaps. Solutions in this case include; building their own memory-efficient mutable Set, Map and List data structures; and building primitive collections for the List, Set, Stack, Bag and Map data structures.
Raab and Mehmandarov then described the challenges developers face today where, for instance, large CSV data needs to be processed in memory. They asked the questions: "How can that efficiently be accomplished in Java?," "How can memory efficiency of data structures be measured?," "What decisions affect memory efficiency?," and "Which approach is better: row vs. columns?" To measure the cost of memory in Java, Raab and Mehmandarov introduced the Java Object Layout (JOL), a small toolbox to analyze object layout schemes in JVMs, and how to use it within an application.
Using a large CSV data set as an example, Raab and Mehmandarov provided a comprehensive look into the various memory considerations: boxed vs. primitive types; mutable vs. immutable data; data pooling; and row-based vs. column-based structures. They also explored three libraries: Java Streams, introduced in Java 8; Eclipse Collections, invented by Raab; and DataFrame-EC, a tabular data structure based on the Eclipse Collections framework. Three Java projects: Amber, Valhalla and Lilliput, are working to improve productivity, value objects and user-defined primitives, and reduce the object header to 64 bits, respectively.
A Bicycle for the (AI) Mind: GPT-4 + Tools was presented by Sherwin Wu, member of the technical staff at OpenAI, and Atty Eleti, a software engineer at OpenAI. In 1973, efficiencies in cycling started to emerge which were compared to a condor bird. Wu and Eleti coined the phrase "A bicycle for the mind" and used this as a metaphor for Steve Jobs and the creation of Apple. In 2023, the emergence of ChatGPT evolved the phrase to "A bicycle for the AI mind."
They discussed large language models (LLMs) and their limitations followed by an introduction to the new function calling capability that improves their gpt-4 and gpt-3.5-turbo applications. Wu and Eleti provided numerous demos for: converting natural language into queries; calling external APIs and multiple functions; and combining advanced reasoning with daily tasks. They maintained that technology is still in its infancy and they are excited to see how this technology will evolve in the future.
Implementing OSSF Scorecards Across an Organization was presented by Chris Swan, an engineer at atsign. Swan introduced the Open Source Security Foundation (OSSF or OpenSSF), a "cross-industry organization that brings together the industry's most important open source security initiatives and the individuals and companies that support them."
The OpenSSF Scorecard project, just one of the projects under OpenSSF, helps open source maintainers improve their security best practices and to help open source consumers judge whether their dependencies are safe. A number of important software security heuristics are measured against an open source software project and assigned with a score of 0-10. A badge is generated containing the heuristic and score that may be placed on the GitHub repository. This provides a visual representation that the maintainers of the open source repository care about security, and a feeling of safety and security.
Swan explored the five holistic security practices: code vulnerabilities, maintenance, continuous testing, source risk assessment and build risk assessment. He encouraged developers who are new to OpenSSF to use Allstar, another OpenSSF project, as a starting point for assessing security for their own open source projects. Swan provided a comprehensive introduction on how to get started by exploring tools such as: GitHub Insights, a tool that can guide developers to create a good quality open source repository; and Terraform, an Infrastructure as a Service (IaaS) resource that provides scripts to improve a GitHub repository.
The process also includes a very long questionnaire in which developers should budget at least one hour. The 80:20 rule in OpenSSF states that 20% of the effort is required to obtain 80% of the Scorecard score. However, Swan commented that it gets more difficult from there and that it is really hard to achieve high scores.