Search stuff
asdf
It takes in CSV files that are created by our Ruby scripts and search and displays the content of chats.
Wrote scipts to process data to csv Wrote logic to parse csv Wrote view logic
asf
Search
CSV Parsing. Don't use Dictionary of Dictionary of Strings.
A short life on GitHub.
Note: these comments are in addition to the main post above. Please read.
The viewer was written in C# and existed to make managing the extracted data easier. Objects were linked between the three tables, allowing sender information to be displayed along side the message. Searching was also implemented, allowing users to search past conversation. Photos were displayed as links to the google plus albums they existed in.
Log in or sign up for Devpost to join the conversation.
Project was split into two halves: a set of data extractors and a way of browsing the extracted data. The data extractors were written in Ruby in order to parse the JSON object returned by Google Takeouts (Hangouts.json). JavaScript engines refused to parse it as the size increased (one of our developers had a ~400MB Hangouts.json object which resulted in a 5GB object in memory). A similar Facebook extractor was written, except their output format was a HTML document with many sibling elements, and parsing that was easier and required less memory. Both extractors wrote out to the same format of three CSVs: One for conversations, mapping conversation IDs to participant user IDs, one for user ID to name mapping, and one containing all the messages with message IDs, conversation IDs, and sender IDs.
Log in or sign up for Devpost to join the conversation.
So it works?
Log in or sign up for Devpost to join the conversation.
Leave feedback in the comments!
Log in or sign up for Devpost to join the conversation.
Log in or sign up for Devpost to join the conversation.