One of the usual drawbacks of a large-scale analysis of data is that details about the individual pieces of data are lost. We have to make the data fit into a specific mold in order to run our program, so we strip the individual pieces of data of their unique elements–the things that make them interesting.
These decisions have to be made with care. How can we maintain the integrity of our data while still making it usable for large-scale analysis? And then, how do we recover those unique elements so that our data pieces can retain their individuality and interest?
I’d like to talk about these issues. If you’ve dealt with this issue before, what was your decision-making process? What are best practices for such things?
