Go at it gently mate and hopefully nothing will fall down the chimney. Remember though, if the people downstairs are using it you'll get smoke/fumes in your place. Get a piece of ply wood or similar bigger then the hole you plan to make. Fit it to the chimney wall over the spot where you're going to make the hole with screws and rawl plugs. Take it off, cut a hole in the ply and fit a duct spigot into it. Make your hole in the wall then fit the board over it. Bang the duct onto the spigot and you're good to go.
Hope I don't come over like teaching grandma to suck eggs!
Hope I don't come over like teaching grandma to suck eggs!