24933 – Mining of Massive Datasets

Presentation

Finding patterns in large datasets is one of the main tasks that a data scientist performs professionally. Data mining sits at the intersection of databases and statistics, and includes several steps from managing to pre-processing, cleaning, modeling, and performing inferences using data.

Data mining can be a challenging task. Data may not be formatted ideally for a purpose, or it may include noisy or missing data points. Datasets can be extremely large making even quadratic-time algorithms impractical. In many cases, the size of a dataset is unbounded and one needs to provide answers as new data elements keep arriving.

This course offers the students the possibility of learning fundamental data mining algorithms.

Associated competences

Basic competences

CB3. That the students have the ability of collecting and interpreting relevant data (normally within their study area) to issue judgements which include a reflection about relevant topics of social, scientific or ethical nature.

Transversal competences

CT3. Applying with flexibility and creativity the acquired knowledge and adapting it to new contexts and situations.

Specific competences

RA.CE7.2 Recognizing the statistic techniques applied to data mining.

RA.CE9.2 Recognizing and applying data mining techniques.

Results from learning

At the end of the course, the students would have acquired:

• Knowledge of typical data mining pipelines.
• Techniques for pre-processing data.
• Knowledge of similarity metrics.
• Techniques for fast similarity searches.
• Knowledge of basket analysis basics.
• Methods for association rules mining.
• Knowledge of recommender systems basics.
• Methods for creating recommender systems.
• Knowledge of the streaming data model.
• Methods for handling data streams and time series.

Requirements

The course requires:

1. Skills in programming and data structures.
2. Basic knowledge of statistics.
3. Skills in machine learning methods.

The practical part of the course will be delivered in Python, hence it is strongly recommended to have a background in Python.

Contents

There are 29 theory lectures (TT01-TT29) in this course.

• TT01 Introduction to Data Mining
• TT02: Data, Methods, and Scenarios
• TT03: Data Preparation - Data Types
• TT04: Data Preparation - Integration and Cleaning
• TT05: Data Preparation - Reduction and Transformation
• TT06: Similarity - Numerical Data
• TT07: Similarity - Beyond Numerical Data
• TT08: Finding Near-Duplicates
• TT09: Locality-Sensitive Hashing
• TT10: Locality-Sensitive Hashing (Additional)
• TT11: Itemsets
• TT12: Association Rules
• TT13: Association Rules Mining
• TT14: Improved Association Rules Mining
• TT15: Extending Association Rules Analysis (Additional)
• TT16: Recommender Systems
• TT17: Recommender Systems - Interaction-Based
• TT18: Recommender Systems - Latent Factors
• TT19: Outliers - Extreme Values
• TT20: Outliers - Probability Density Methods
• TT21: Outliers - Density- and Isolation-Based Methods
• TT22: Data Streams
• TT23: Data Streams - Reservoir Sampling
• TT24: Data Streams - Bloom Filters
• TT25: Data Streams - Probabilistic Counting
• TT26: Data Streams - Estimating Moments (Additional)
• TT27: Time Series Analysis
• TT28: Time Series - Similarity
• TT29: Time Series - Forecasting

Methodology

The course is structured around theory classes in which the topics of the course are introduced.

In seminar and practice sessions, students can work individually or in small groups in performing network analysis tasks. At the end of each session, each student reports his/her findings individually with a 1-2 pages report.

There are three group projects (groups of two students), which integrate the techniques seen in the seminar sessions. These are accompanied by practice sessions for instructors to solve student questions regarding these projects.

Evaluation

See evaluation rules

Bibliography

• Data Mining, The Textbook (2015) by Charu Agrawal. ISBN 978-3-319-14142-8. Free Download
• Mining of Massive Datasets SECOND EDITION (2014) by Leskovec et al. ISBN 978-1107077232. Online materials: http://www.mmds.org/. Free Download
• Introduction to Data Mining SECOND EDITION (2019) by Tan et al. ISBN 978-0-13-312890-1. Online materials: https://www-users.cs.umn.edu/~kumar001/dmbook/index.php
• Data Mining Concepts and Techniques THIRD EDITION (2011) by Han et al. ISBN 978-0123814791.