Not really, if you cut it up into half-inch lengths or smaller , it should pass through you. Fibre is essential in any diet, as it helps the passage of other foodstuffs. In fact, when I next harvest I will have a go and tell you about it - I can't imagine it will taste too good without some other flavoring!
I've never used the stalks as an edible, but its cellulose content may be hard to digest. You could try a cup or four in a vegetable stock and use the stock in whatever kind of soup you want, though.