Just wanted to update everyone on this issue. I will also post a new thread with my solution. I went up to a size 40 pilot jet today after diagnosing the issue with the carburetor. The problem is Polaris installs too small of a pilot jet from the factory (almost makes you wonder if they pulled this carb off of a lawnmower and slapped it on the RZR without testing it.) Based on the number of 170 owners who have this issue, I can almost guarantee that this is the issue.
When testing the pilot circuit, you are supposed to turn the Pilot jet in (richens the fuel mixture) until the engine starts to cut out. If you turn it all the way in, and there is no change in the idle condition, then your pilot is too small. I did this test with both a 35, 38, and 40 pilot jet. The 38 pilot jet had to be turned all the way closed, and then brought back a 1/4 turn to achieve the correct fuel mixture. This is not ideal, because it doesn't leave you room for further adjustment, and it's not as efficient. The 40 Pilot is set where you want your pilot circuit to usually sit, which is 2 1/2 turns out from closed. When doing this, I put everything back together, and had my son ride it up and down the street, with several WOT runs with coming to a dead stop, and where it would typically sputter, spit, and backfire with the stock pilot, it did not with the 40 Pilot.
So, if you want your RZR170 to run the way it should, and not too lean on the pilot circuit from idle to 1/4 throttle, you should pick one of these Pilot jets up from your dealer and swap it out. It will make a huge difference in the way these little guys run.
http://www.rzrforums.net/rzr-170-specific-disccusion/50914-solution-backfiring-170-issue.html