For the CultureInfo constructor overloads you really want to use names instead of the int/LCID. Names allow you to access custom cultures that a user may install for future versions, and will be more extensible in the future. You also really want to use user overrides. User overrides are there because the user did not like the defaults, or because they are inappropriate or ambiguous for the user's environment.
Always use cultures when appropriate, but be careful where you use them. They are meant to be specific for a user or machine, and they could change. For example, as countries join the EU, their currency symbols or formats change. In other cases, companies or users may prefer a 24-hour clock to a 12-hour default. Culture data is necessary and great when presenting data to the humans who are actually using the computer, but it's an awful way to store data. If you have to read something back later, save it in a binary format, use CultureInfo.InvariantCulture, or specify a specific format to the formatting functions. Except for Invariant, don't expect the culture data provided by this class to remain the same, even between instances running as the same user.
CultureInfo is a useful encapsulation of all the culture-specific data that you'll need for your operations. It can be used to control string comparisons, number formatting, date and calendar formatting, and resource lookups. See Thread's CurrentCulture and CurrentUICulture for ways of setting this culture on your current thread. It is also one of the most interesting types that implement IFormatProvider. The ToString method on our numeric types and DateTime will call GetFormat(typeof(NumberFormatInfo)) or GetFormat(typeof (DateTimeFormatInfo)) for formatting and parsing.
We were never quite sure we got the balance between this type and RegionInfo correct. RegionInfo is very commonly overlooked, when you'd expect that perhaps some of the features on this class would be on RegionInfo instead.
In future versions of the Common Language Runtime, look for custom culture support and custom resource fallback support. Custom culture support will allow users to define their own cultures in an easy manner that the entire system will respect.
For the System.Resources.ResourceManager, we hope to eventually add in support for fallback along more axes beyond culture, like age (for different scripts like Hiragana or Katakana), gender, and perhaps region or ZIP code (e.g., for different phone numbers).
One of my biggest regrets is that CultureInfo.ClearCachedData() is an instance method rather than a static method. This method actually clears shared (static) caches for system settings like CurrentCulture, CurrentRegion, and the like. It does nothing to individual instances, so it is very confusing to have this as an instance method. Unfortunately, this was discovered very late in the .NET Framework cycle, and it was considered a breaking change to make it static, so it had to ship this way.