Automatic Machine Learning (AutoML) is an area of research aimed at automating Machine Learning (ML) activities that currently require the involvement of human experts. One of the most challenging tasks in this field is the automatic generation of end-to-end ML pipelines: combining multiple types of ML algorithms into a single architecture used for analysis of previously-unseen data. This task has two challenging aspects: the first is the need to explore a large search space of algorithms and pipeline architectures. The second challenge is the computational cost of training and evaluating multiple pipelines. In this study we present DeepLine, a reinforcement learning-based approach for automatic pipeline generation. Our proposed approach utilizes an efficient representation of the search space together with a novel method for operating in environments with large and dynamic action spaces. By leveraging past knowledge gained from previously-analyzed datasets, our approach only needs to generate and evaluate few dozens of pipelines to reach comparable or better performance than current state-of-the-art AutoML systems that evaluate hundreds and even thousands of pipelines in their optimization process. Evaluation on 56 classification datasets demonstrates the merits of our approach.