A second straight league win to start the season took Celtic above St Mirren on goal difference at the top of the early Scottish Premiership table.
O'Neill, 74, missed the trip to Rugby Park after a "small procedure" in hospital earlier this week.
Celtic raced 3-0 ahead before half-time thanks to a treble from Hogh, the Glasgow giants' club-record signing after joining from Bodo/Glimt for a reported £11 million ($15 million).
"I'm so happy right now. It was a good feeling, but the most important thing was how we did as a team, and it was a crazy, good game," Hogh, who failed to score on debut against Dundee, told Sky Sports.
"I said I would take one day at a time and try to do better."
Celtic assistant manager Shaun Maloney said, "I said before, it's the life of a striker. I think you're there, you score some, you miss some. He was really good today."
Hogh needed less than a minute on Sunday to open the scoring, heading in at the back post from a cross by Arne Engels.
Hogh doubled Celtic's lead in the 33rd minute, with Camilo Duran seizing on Johnly Yfeko's pass out of defence.
He combined with Benjamin Nygren to work the ball to Hogh on the left edge of the box, with the forward finishing well.
Two minutes later, Hogh had his hat-trick.
Nygren slipped the ball right for Duran, whose cross took Kilmarnock goalkeeper Max Stryjek out of play, with Hogh nodding into an empty net.
Celtic extended their lead early in the second half when they were effectively gifted a fourth goal by Stryjek after his pass went straight to Nygren in front of the goal.
Michael Schjonning-Larsen pulled a goal back for Kilmarnock in the 56th minute.
But Celtic were four goals ahead again when substitute Luke McCowan, moments after coming on, volleyed in from close range 15 minutes from time after Kilmarnock made a mess of trying to clear a corner.
Kilmarnock have conceded nine goals in their opening two league games.
Later Sunday, last season's runners-up Hearts host Dundee United as they look to bounce back from their opening-day defeat by Aberdeen, while Rangers face Hibernian at Ibrox.