CEO Dave Kimbell is stepping down effective immediately, Ulta said Monday, and will stay on as an advisor through June 28.