This gets quite fiddly since dashboard apps aren't identified by a single app label. We need to special-case them in a few places.