Last week, the UK Guardian newspaper reported that security researchers had discovered a log of locations being locally cached on an iPhone 4, from a report posted at O'Reilly.
However, this information was not new, as not only had it been known about for some time (together with an iOS 3 version), it was explicitly stated in Session 115 of WWDC 2010 “Usign Core Location in iOS” around the 13:45 time. This is a feature of the platform which aids in assisting quick location resolution, not a tracking system, which has been well known about for a while.
As a result of the widespread media attention, and the threat of a senate privacy request, Apple has answered questions regarding the use of the location data. They confirm that the data is not tracking the user's direct location, but rather downloading nearby WiFi spots in case you happen to be nearby.
However, they do confirm the existence of a few bugs. For example, when location services are off, the device still collects information in the cache. In addition, the cache is not cleaned and thus continuously builds up an imprint of the areas in which you have been both recently and in the past.
Apple confirms that in future the cache will be cleaned for a 7 day limit, and that the data will no longer be part of the backups collected by iTunes. The next major release of iOS will encrypt the cache so that even if the phone is compromised, the data will not be accessible.