Yes, you can compost celery.
Unwanted stalks, and sliced off tops and bottoms, can be all composted – though don’t forget that not so crunchy celery sticks can still be a useful addition to a stock pan or soups/casseroles.
Don’t compost any sticks that have been heavily coated in salad dressing or the like – the oils or other flavourings in the dressing might attract rodents or other unwanted visitors to your heap.
In a compost heap, celery will rot down fairly quickly — it’s quite tough in some ways but also water-heavy so the composting microbes will be happy enough. It’s a green in composting terms – if you’ve got a lot of it, add it alongside browns (like cardboard or fallen leaves) to keep you heap balanced.