Hindsight =

Datamining

Hindsight is learning from past mistakes in order to build expertise. Hindsight involves critical reflection in order to integrate knowledge gained from the past with other sources.

Qualitative data mining refers to computational methods to gather, process and analyze large quantities of textual data (film scripts), to extract and exploit interesting patterns.

ScriptBook trained its algorithms on a large historical dataset of film scripts which have had a cinematic release between 1970 and 2016, making it possible to detect patterns and extract features that compelling storylines have in common.

Foresight =

Machine Learning + Natural Language Processing

Machine Learning is about foresight. Machine Learning comprises of a set of tools that allow computers to "learn", by examining data about past cases with known outcomes (called “training data” or “previously released movies”).
Using this data it builds a model that can be tested by making predictions on other known cases (a set of released movies left out of the training process). The final model can be used to predict the commercial and critical success of new film scripts.

Natural Language Processing (NLP) deals with making a machine "understand" the structure and the meaning of written text. Using a combination of statistical strategies and external sources of knowledge, it is possible to extract semantics and topical concepts from a raw stream of words, which encode valuable information about the content. Applying NLP algorithms on scripted narratives such as a screenplays is challenging because relevant information often depends on subtle and complicated variables, including slang, subtext and social context.

Insight =

Feature Engineering

Insight is only possible when we put hindsight and foresight together to create knowledge that can be used for present decision making. Feature Engineering is an art, like writing is an art, like acting is an art. At ScriptBook, we spend most of our efforts in engineering story-based features (parameters).
Story-based parameters have to be created manually. This requires spending a lot of time with actual sample data (film scripts) and thinking about the underlying form of the problem, structures in the data and how best to expose them to predictive modeling algorithms.

Within ScriptBook our feature selection algorithms use a scoring method to rank and choose features that correlate well with commercial or critical success. Engineering meaningful representations of narratives requires a unique blend of machine learning, NLP and screenwriting domain knowledge. It also is ScriptBook’s secret sauce.