There are some videos on replacement of the mirror. I did this on my car. I had to buy tool to assist with interior door trim. This allows you to unclip all the clips behind the skin without marring anything. I also purchased tool to remove manual window door crank, as I don't have power windows. Once the interior door skin is off, it is much easier to swap out the mirror. I left mine in the black color, as that is how my 2003 CE is. On another Corolla I assisted with (2006 S), we also left it in black.

You can probably paint it without primer, if you use plastic prep. Then spray on your base coat, followed by clear coat.