If it's Polygons, you can use Vector Render
using render layers.
Set up your scene.
1. Choose Render Layers from the Layers Menu.
2. Select EVERYTHING in your scene (lights, camera, models, etc.) then click on Create New Layer and Assign Selected Objects from the render layers menu.This will give you a render layer with all objects in it. Rename this layer to BeautyPass. The BeautyPass layer will be rendered normally.
3. Now select only the object you want the wireframe applied to and click the Create New Layer and Assign Selected Objects again. This will give you a render layer with only the selected object. Rename this layer to Wireframe.
4. Hypershade - create a Black Lambert and apply it only to the object in the Wireframe render layer.
When you use render layers, it will show an 'R' next to the layer indicating whether it will be rendered or not.
Normally, it will render all layers marked with the 'R' in one shot. You will need to manually set the render option for each layer so that they render individually. This is easy.
1. Set up your render option normally (software, hardware, mental ray, etc.)
2. Turn off the R option in the Wireframe render layer so that it will not render.
3. Render the BeautyPass layer.
1. Turn off the R option for the BeautyPass layer and turn it ON for the Wireframe layer.
2. In the Render Options choose May Vector. If you do not see it in the list, go to Window > Settings Preferences > Plugin Manager and check the box next to VectorRender.mll. You should now see it appear in the Render options window.
3. Choose the Maya Vector tab.
4. Under 'Edge Options' check the box for 'Include Edges'.
5. Set Hairline and Edge Weight to preference but leaving them set to default works very well.
6. Edge Style should be set to Outlines if you want Quads to show. Changing it to Entire Mesh will give you triangles.
7. Change the Edge Color to anything but black. White works well since you can change the color in After Effects or your favorite compositng program.
8. Close out the render settings window.
9. Right click over your model and choose Edges then select the entire object so that all of its edges are selected.
10. Under the Polygons Menu Set choose Normals > Harden Edge - this is what lets you render the entire wireframe. Not doing this will cause it to render only an outline around your model, kind of like what a Stroke does in Photoshop.
11. Render your scene.
You can use any compositor you wish, I will explain this using After Effects.
1. Import your animation files.
2. In the timeline, place the Wireframe layer over the BeautyPass layer.
3. Switch to Modes options in the timeline window.
4. On the Wireframe layer change the blend mode to Screen. This will cause the black to disappear and leaving only the wireframe.
This method gives you full control over the look of the wireframe. From here, you can change the color and opacity and composite it over either the BeautyPass or a smooth shaded, low-poly version as well. Be creative with how you compoite it as well. Masks create endless possibilities!
Hope that helps.