It could be the valve seals, valve guides or leaking intake gaskets.
You maybe able to use a vacuum gauge and find out more.
The link I'm including you'll notice at the bottom of the page it will have Scenario 1 to 15. By clicking on them it will show what the vacuum gauge will do with a specific engine issue so you'll get a better idea on what the problem is.
How to Use and Interpret a Vacuum Gauge