Microbes in marine sediments represent a large portion of the biosphere, and resolving their ecology is crucial for understanding global ocean processes. Single-gene diversity surveys have revealed several uncultured lineages that are widespread in ocean sediments and whose ecological roles are unknown, and advancements in the computational analysis of increasingly large genomic data sets have made it possible to reconstruct individual genomes from complex microbial communities. Using these metagenomic approaches to characterize sediments is transforming our view of microbial communities on the ocean floor and the biodiversity of the planet. In recent years, marine sediments have been a prominent source of new lineages in the tree of life. The incorporation of these lineages into existing phylogenies has revealed that many belong to distinct phyla, including archaeal phyla that are advancing our understanding of the origins of cellular complexity and eukaryotes. Detailed comparisons of the metabolic potentials of these new lineages have made it clear that uncultured bacteria and archaea are capable of mediating key previously undescribed steps in carbon and nutrient cycling.