Usage of data generated by drones in big data and operational analytics is going to add a new horizon in data storage, transfer infrastructure and data processing software, according to Kirk Borne, professor of Astrophysics and Computational Science at George Mason University.
Big data generated by drones is useful in every sector including monitoring data of animal cruelty on farms and surveillance data from military drones of distant lands and adversaries. Kirk recently talked about the challenge of processing, storing and transferring this data.
Kirk gave a reference of Hank Hogan article about the US military, which depicts the need to increase their storage capacity for the surveillance data from drones. Kirk mentioned that massive data growth and processing of data creates the massive technology challenges for big data and operational analytics, which include various data mining scenarios and mining the data from Internet of Things.
The video feed data storage and processing bottleneck from drones represents just one more example of massive data growth creating massive technology challenges. These challenges fall under the umbrella of operational analytics, which includes mining machine logs in operational settings.
Hank shared some statistical data from military sources The Air Force Intelligence, Surveillance and Reconnaissance (ISR) Agency collects about 1,600 hours of video per day. A single 14-hour Gorgon Stare drone mission generates about 70 terabytes of data. Hank mentioned the need for a dynamic infrastructure, with systems and storage assembled as needed to satisfy a mission.
Colin Snow, senior director at SAP said that the drones’ usage needs a revolution in big data cloud services. He also said that flying a drone and taking pictures is merely the first step in the data collection process. The images need to be corrected, calibrated, processed, stored, and evaluated. He mentioned PrecisionMapper, a cloud-based application that gives anyone the ability to upload, store, process, and share her aerial image data.
Certainly the use of a cloud-based in-memory computing platform to accelerate analytics, processes, and predictive capabilities will be foundational to that differentiation.
Mark Underwood, who works in the field of big data security, wrote about encryption and compression of data collected by drones. Mark says that since software to intelligently reason directly from video feeds is still only in a research phase, drone data handling needs improvements, which include the increased on-board processing, encryption, compression and fusion of data.
Kirk states Apache Spark as just the beginning of real solution for the challenge of real-time big data analytics. He also mentioned that MapR has started investigation on the data challenges, communication standards, and analytics requirement related to this problem. He collated the common characteristics of challenging problems as:
- High input rate
- Streaming (time-series) data
- Many small files
- Need for fast micro-adjustments in the operational environment (including supply chain replenishment, just-in-time delivery, event response, and more)
Kirk also talked about the possibility of increased sales and revenue for Amazon, which recently proposed delivery of food using drones.
Amazon is proposing to deliver food shipments to your home by way of drone: “air-to-surface” supply replenishment deliveries! ……. Amazon can use their customer data to make offers and recommend other products to you prior to the delivery of your food shipment, such as: “special discount on your new HDTV if you order it in time for your next food shipment.” The potential for increased sales and revenues for Amazon would be astronomical ….
By gathering data on a large scale over time, service providers will be able to process unprecedented levels of detail data and turn it into usable information. Ernest Earon, president at PrecisionHawk, says the company views itself as a data company rather than a drone company. He envisions an “app store” model that would allow, say, somebody in North Dakota with a top-notch algorithm for detecting potato blight to license it to other farmers.