Comfortable and maybe trendy/edgy but definitely not "dressy."
In my experience performing and/or waiting around for my kids who are performing, backstage generally means a lot of sitting around and waiting. A big venue will likely have a dedicated room with seating for you and possibly refreshments. A smaller venue might have a dusty old greenroom with some old sofas and a water fountain. Either way, comfort is key.
I arise in the morning torn between a desire to improve the world and a desire to enjoy the world. This makes it hard to plan the day. ~E.B. White