Exploring Big Data

Inspiration

Big Data and its use to solve Real World Problems.

What it does

Exploring Big Data and how it is used in the Real World.

What we learned

When Data collected is uncompressible by using Traditional methods, it is said to be Big Data. Big Data is characterised by

• Volume: large amounts of data ranging in petabytes or even more
• Velocity: the speed at which this data is generated, i.e. device like IoT can generate terabytes of data in just one day
• Variety: the type of Data i.e the format of data, it may be structured, unstructured, classified in distinct quantities or relatively unquantified.

Additional Characteristics
• Value: knowing what we want to extract from data.
• Variability: the reliability of the Data received; biases, source faults etc

The main benefits if Big Data are dependant not mainly on the Volume or the size of Data a company or model processes, but How Big Data is processed? Analysis extracted from the Big Data leads to better User-Behaviour understanding, increased security, fraud prevention, maximising opportunities and rethinkng entire risk portfolios in mere seconds.

The implementations of Big Data are completely different from Relational Databases, i.e they are usually stored in Data Lakes and on the Cloud considering its size. Distributed Strorage and Clusters are used to store and compute on this data. Various different frameworks to manage Big Data have come up such as MongoDB, Hadoop etc.

The steps in managing and using the Data that is collected are

• Ingestion
• Storage
• Pre-processing
• Processing

What's next for Exploring Big Data

Exploring more use cases, learning how to implement it.

Built With

• bigdata