Du,

These are gross failures >90 degrees in the affine registration step. Typically this comes from input .nii files where the orientation information (sform, qform) are not accurate. This may have been caused by your cropping program or any other pre-processing. Typically raw DICOM or DICOM converted with dcm2niix would not have this issue. You can confirm by opening the .nii with an image viewer that shows A/P S/I L/R labels, and verifying that they match the image itself. 

1: ANTs nonlinear is always used regardless, but your failure here is in the initial affine registration, before the nonlinear. reg_aladin from niftyreg is always used for affine registration, unless you provide your own registration parameters file.

2. Two masks are defined in the template space. One prioritizes the face region in the registration cost function, and includes the whole head. It uses the one with the lowest cost function, between the two. altReg=1 tells it to use the result from the mask that had the higher cost function, instead. Rarely, this fixes smaller registration issues, but your issues are coarser than this is likely to fix.

3. You should provide inputs in native space, and an affine registration to the MCALT_FaceTemplate space. You shouldn't resample the inputs to that space ahead of time. If you use native-space input images, the outputs will also be in native space. They will always match the voxel spaces of the input file.

Let me know if you have further questions.
Best regards,
Chris Schwarz