Any cloud provider that believes in data gravity is trying to make it easier to collect and store data in its facilities. To make data movement between cloud and on-premises endpoints easier, Microsoft recently announced the general availability of Azure Data Factory (ADF).
At QCon San Francisco, we offer two days of workshops (Nov 19-20). Workshops focus on developing the technical skills that leverage technologies you heard about from our expert practitioners during the conference sessions. Here is a glimpse at some of the experts you can learn from QCon SF ‘15 workshops.
Snowflake Computing has announced the general availability of their Snowflake Elastic Data Warehouse, a software as a service offering that provides a SQL data warehouse on top of Amazon Web Services.
AWS updated DynamoDB with the ability to publish near real-time notifications of data changes. This new capability – called DynamoDB Streams – spawned two additional features for the NoSQL database-as-a-service: DynamoDB Triggers fire based on specific data changes found in a DynamoDB Stream, and cross-region replication is driven by a DynamoDB Streams-based architecture.
IBM has announced a new web portal called developerWorks Open, bringing together various projects they are open sourcing. The projects cover many domains including Analytics, Cloud, IoT, Mobile, Security, Social, Watson and others. So far, IBM has open sourced about 30 projects, and they plan to increase the number up to 50 by the end of the year, and others may come in the future.
For an organization to be data-driven, it's not enough to just dump mountains of data. That data needs to be accurate and meaningful. Julianna Göbölös-Szabó, data engineer at Prezi shared how they improved the quality of its log data. Their solution involved moving from unstructured to structured data with a lightweight, contract-based approach to nudge all teams in the right direction.
SQL Server 2016 offers a new tool for performance tuning called Query Store. This holds metrics that let you quickly see when an execution plan change has negatively impacted the database.
A database query times out and you don’t know why. The estimated query plan is revealing the problem, so you remove the timeout entirely. An hour later it is still running and you are no closer to getting the actual execution plan. If only there was a way to find out what is actually happening inside the server. With Live Query Statistics in SQL Server 2016 you can now answer that question.
Basho Data Platform supports integration with NoSQL databases like Redis, in-memory analytics, caching, and search. Basho Technologies, the company behind Riak NoSQL database, announced in May, the availability of the data platform that can be used to deploy and manage Big Data, IoT and hybrid cloud applications.
Only 38% of IT professionals use containers in production environments, according to a recent survey. ClusterHQ, which ran the survey of the current state of container usage and adoption, also concludes that 73% of respondents are running containers in a VM environment.
At the recent devopsdays Amsterdam 2015, Patrick Roelke contended that monitoring still has lots of issues. Roelke believes that data science can help by eliminating static thresholds and coalescing information from various data sources into a single metric. The talk included a quick overview of monitoring tools that leverage data science: Kale, Bosun and AnomalyDetection.
Metanautix recently announced the newest version of its product, Quest. Quest allows enterprises to build software defined data marts that can run in virtualized servers. Designed from the ground up with security and auditability in mind, Quest can deal with Big Data workloads and encapsulate it into different logical views, ready for consumption by different departments in enterprise.
The demand for IT project managers is increasing. Agile methodologies support collaboration with distributed teams for creative problem solving. The Internet of Things, cloud, big data, and cyber security will continue to dominate the IT landscape. Project managers have to pioneer IOT initiatives, be prepared for the influx of data and ensure that deliverables from their projects are secure.
In order to improve scalability, Parse moved part of their services, including their API, from Ruby on Rails to Go, Charity Majors, Engineer at Parse, recounts. In doing so, both their reliability and deployment times benefited greatly.
Data masking is a necessary, but error prone process. You only need to forget the mask one time to leak sensitive data. SQL Server 2016 attempts to address this with a feature called Dynamic Data Masking.