Counterfeiting of physical goods is a global problem amount-ing to nearly 7% of world trade. While there have been a variety of overt technologies like holograms and specialized barcodes and covert technologies like taggants and PUFs,these solutions have had a limited impact on the counterfeit market due to a variety of factors - clonability, cost or adoption barriers. In this paper, we introduce a new mechanism that uses machine learning algorithms on microscopic images of physical objects to distinguish between genuine and counterfeit versions of the same product. The underlying principle of our system stems from the idea that microscopic characteristics in a genuine product or a class of products (corresponding to the same larger product line), exhibit inherent similarities that can be used to distinguish these products from their corresponding counterfeit versions. A key building block for our system is a wide-angle microscopy device compatible with a mobile device that enables a user to easily capture the microscopic image of a large area of a physical object. Based on the captured microscopic images, we show that using machine learning algorithms (ConvNets and bag of words), one can generate a highly accurate classication engine for separating the genuine versions of a product from the counterfeit ones; this property also holds for super-fake counterfeits observed in the marketplace that are not easily discernible from the human eye. We describe the design of an end-to-end physical authentication system leveraging mobile devices, portable hardware and a cloud-based object verication ecosystem. We evaluate our system using a large dataset of 3 million images across various objects and materials such as fabrics, leather, pills, electronics, toys and shoes. The classication accuracy is more than 98% and we show how our system works with a cellphone to verify the authenticity of everyday objects.