Activated alumina is aluminum oxide, the same chemical substance as sapphire and rubies, but without the impurities that give the gems their color.
(This is not aluminum, which is a more highly-processed metal made from activated alumina.)
Activated alumina has a very high surface-area-to-weight ratio, with many tunnel-like pores that run all through it. This media features high surface area against which the water interacts, thereby removing fluoride (and arsenic). Through the attraction of the metals, 90%+ of fluoride (and arsenic) is normally removed. The fluoride coats the surface of the alumina over time and eventually the cartridge needs to be replaced (usually once a year for a family of four).