Deep Learning Day
at KDD

Monday, August 5, 2019 Stay Informed

Deep Learning Day

The impact of deep learning in data science has been nothing short of transformative. Powered by the surge in modern computation capabilities, widespread data availability, and advances in coding frameworks, deep neural networks are now ubiquitous. Deep methods yield state-of-the-art performance in many domains (computer vision, speech recognition and generation, natural language processing), and are still widening their lead as more research appears daily. As the field has matured, it has witnessed a lot of theoretical and practical advances. Recently, there has been a shift towards more rigorous and robust experiments, and more interpretable and transparent theory and models that can further help understand the great empirical successes that a great deal of real-world applications have been enjoying.

At KDD 2019, the Deep Learning Day is a key event, specifically dedicated to deep learning’s impact on data science. The goal is to provide an overview of recent developments in deep learning, particularly for data mining. During the day, we will welcome many diverse and exciting invited speakers from world class research institutions as well as deep learning-themed workshops focusing on emerging topics in KDD.

In summary, the Deep Learning Day at KDD 2019 will include a broad range of activities - a plenary half day with an exciting lineup of plenary speakers and a half-day of deep learning themed workshops. On behalf of the Deep Learning Day and KDD 2019 organizing committee, we welcome you all to attend this event!


8:00AM - 12:00PM Deep Learning Day Plenary Keynotes [La Perouse, Street Level, Egan]

  • 8:00AM - 08:45AM Dawn Song (UC Berkeley)—- AI and Security: lessons, challenges and future directions
  • 8:45AM - 09:30AM Jie Tang (Tsinghua University)—- Graph Embedding and Reasoning
  • 9:30AM - 10:00AM Coffee Break
  • 10:00AM - 10:45AM Ruslan Salakhutdinov (CMU & Apple)—- Deep Learning: Recent Advances and New Challenges
  • 10:45AM - 11:30AM Jure Leskovec (Stanford & Pinterest)—- How Powerful Are Graph Neural Networks
  • 11:30AM - 12:00PM Deep Learning Day Panel with Dawn Song, Jie Tang, Ruslan Salakhutdinov, and Jure Leskovec

12:00PM - 1:00PM Lunch

1:00PM - 5:00PM Deep Learning Day Workshops

  • Workshop on Deep Reinforcement Learning for Knowledge Discovery [Summit 1, Ground Level, Egan]
  • Workshop on Deep Learning Practice for High-Dimensional Sparse Data [Summit 2, Ground Level, Egan]
  • Workshop on Deep Learning on Graphs: Methods & Applications [Summit 10, Ground Level, Egan]
  • Workshop on Deep Learning for Education [Summit 14, Ground Level, Egan]

Plenary Keynote Speakers

Dawn Song (University of California, Berkeley)

AI and Security: lessons, challenges and future directions (Download slides)
In this talk, I will talk about challenges and exciting new opportunities at the intersection of AI and Security,how AI and deep learning can enable better security, and how Security can enable better AI. In particular, I will talk about secure deep learning and challenges and approaches to ensure the integrity of decisions made by deep learning. I will also give an overview on challenges and new techniques to enable privacy-preserving machine learning. Finally, I will conclude with future directions at the intersection of AI and Security.

Jie Tang (Tsinghua University)

Graph Embedding and Reasoning (Download slides)

I will present a theoretical analysis for several recently developed graph embedding algorithms (DeepWalk, LINE, PTE, and node2vec), to show that all the algorithms can be unified into a matrix factorization framework (NetMF) with closed forms. Then I will present a scalable learning algorithm NetSMF, which leverages theories from spectral sparsification to significantly improve efficiency in embedding learning. NetSMF can learn network representations of a billion-edge graph on a single server. Further, we introduce ProNE —a fast, scalable, and effective model, whose single-thread version is 10–400× faster than the above algorithms with 20 threads. Based on the graph embedding results, I will also introduce a novel CognitiveGraph framework for learning with reasoning. CognitiveGraph is founded on the dual process theory in cognitive science. The framework gradually builds a cognitive graph in an iterative process by coordinating an implicit extraction module (System 1) and an explicit reasoning module (System 2). CognitiveGraph can be applied to many tasks on graphs, e.g., multi-hop reasoning-based QA, knowledge graph completion, etc.

Russ Salakhutdinov (Carnegie Mellon University & Apple)

Deep Learning: Recent Advances and New Challenges

In the first part of the talk, I will introduce XLNet, a generalized autoregressive pretraining method for natural language understanding that (1) enables learning bidirectional contexts by maximizing the expected likelihood over all permutations of the factorization order and (2) overcomes some limitations of BERT due to its autoregressive formulation. I will further show how XLNet integrates ideas from Transformer-XL, the state-of-the-art autoregressive model, into pretraining and demonstrate that XLNet outperforms BERT on a number of NLP tasks. In the second part of the talk, I will show how we can design modular hierarchical reinforcement learning agents for visual navigation that can perform tasks specified by natural language instructions, perform efficient exploration and long-term planning, learn to build the map of the environment, while generalizing across domains and tasks.

Jure Leskovec (Stanford University & Pinterest)

How powerful are graph neural networks (Download slides)
Machine learning on graphs is an important and ubiquitous task with applications ranging from drug design to friendship recommendation in social networks. The primary challenge in this domain is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models. However, traditionally machine learning approaches relied on user-defined heuristics to extract features encoding structural information about a graph. In this talk I will discuss methods that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning and nonlinear dimensionality reduction. I will provide a conceptual review of key advancements in this area of representation learning on graphs, including random-walk based algorithms, and graph convolutional networks.


Deep Reinforcement Learning for Knowledge Discovery

Organizers: Jiliang Tang, Dawei Yin, Long Xia, Alex Beutel, Shaili Jain, Minmin Chen and Xiangyu Zhao

While supervised and unsupervised learning have been extensively used for knowledge discovery for decades and have achieved immense success, much less attention has been paid to reinforcement learning in knowledge discovery until the recent emergence of deep reinforcement learning (DRL). By integrating deep learning into reinforcement learning, DRL is not only capable of continuing sensing and learning to act, but also capturing complex patterns with the power of deep learning. Recent years have witnessed the enormous success of DRL for numerous domains such as the game of Go, video games, and robotics, leading up to increasing advances of DRL for knowledge discovery. For instance, RL-based recommender systems have been developed to produce recommendations that maximize user utility (reward) in the long run for interactive systems; RL-based traffic signal systems have been designed to control traffic lights in real time to enhance traffic efficiency for urban computing. Similar excitement has been generated in other areas of knowledge discovery, such as graph optimization, interactive dialogue systems, and big data systems. While these successes show the promise of DRL, applying learning from game-based DRL to knowledge discovery is fraught with unique challenges, including, but not limited to, extreme data sparsity, power-law distributed samples, and large state and action spaces. Therefore, it is timely and necessary to provide a venue, which can bring together academia researchers and industry practitioners (1) to discuss the principles, limitations and applications of DRL for knowledge discovery; and (2) to foster research on innovative algorithms, novel techniques, and new applications of DRL to knowledge discovery.

1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data

Organizers: Xiaoqiang Zhu, Kuang-Chih Lee, Jian Xu, Guorui Zhou, Biye Jiang, Jun Lang, Wenwu Ou, Hongbo Deng, Weinan Zhang and John Canny

In the increasingly digitalized world, it is of utmost importance for various applications to harness the ability to process, understand, and exploit data collected from the Internet. For instance, in customer-centric applications such as personalized recommendation, online advertising, and search engines, interest/intention modeling from customers’ behavioral data can not only significantly enhance user experiences but also greatly contribute to revenues. Recently, we have witnessed that Deep Learning-based approaches began to empower these internet- scale applications by better leveraging the massive data. However, the data in these internet-scale applications are high dimensional and extremely sparse, which makes it different from many applications with dense data such as image classification and speech recognition where Deep Learning-based approaches have been extensively studied. For example, the training samples of a typical click-through rate (CTR) prediction task often involve billions of sparse features, how to mine, model and inference from such data becomes an interesting problem, and how to leverage such data in Deep Learning could be a new research direction. The characteristics of such data pose unique challenges to the adoption of Deep Learning in these applications, including modeling, training, and online serving, etc. More and more communities from both academia and industry have initiated the endeavors to solve these challenges. This workshop will provide a venue for both the research and engineering communities to discuss the challenges, opportunities, and new ideas in the practice of Deep Learning on high-dimensional sparse data.

Deep Learning on Graphs: Methods and Applications (DLG’19)

Organizers: Jian Pei, Lingfei Wu, Yinglong Xia, Hongxia Yang

Deep Learning models are at the core of research in Artificial Intelligence research today. It is well-known that deep learning techniques that were disruptive for Euclidean data such as images or sequence data such as text are not immediately applicable to graph-structured data. This gap has driven a tide in research for deep learning on graphs on various tasks such as graph representation learning, graph generation, and graph classification. New neural network architectures on graph-structured data have achieved remarkable performance in these tasks when applied to domains such as social networks, bioinformatics and medical informatics.

This workshop aims to bring together both academic researchers and industrial practitioners from different backgrounds and perspectives to above challenges. The workshop will consist of contributed talks, contributed posters, and invited talks on a wide variety of the methods and applications. Work-in-progress papers, demos, and visionary papers are also welcome. This workshop intends to share visions of investigating new approaches and methods at the intersection of Graph Neural Networks and real-world applications.

Deep Learning for Education (DL4Ed)

Organizers: Andrew Lan, Byung-Hak Kim, Richard Baraniuk, Michael Mozer and Jacob Whitehill

We are happy to announce that we will be organizing a half-day workshop at KDD 2019, as part of the Deep Learning Day. Details on the workshop will be updated soon. KDD 2019 will be held in Anchorage, Alaska, USA during August 4-8, 2019. See here.

Many people believe in education because of its power to maximize human potential, and transform lives; it is also a critical component of our society, since both school education and lifelong education play a central role in the development of the world’s workforce. However, compared to other applications of significant societal impact (e.g., healthcare, transportation, and social sciences), education remains a highly under-explored application field by the data mining and machine learning research communities.

General Chairs

Image description

Yuxiao Dong

Microsoft Research

Image description

Vagelis Papalexakis

University of California Riverside

How can we assist you?

We'll be updating the website as information becomes available. If you have a question that requires immediate attention, please feel free to contact us. Thank you!

Please enter the word you see in the image below: