Latency issue

When I record audio tracks there is a slight but significant delay. After recording I have to drag the recording up sooner in my timeline to compensate. The strange thing is that I think it only does this with some of my projects.
Where would I look to shorten up this latency?