Detail textures:

Detail textures are laid on a very large grid. In the ME, select Detail Map Editor from the toolbar. You will see a white grid mesh hovering under your cursor in the 3D viewport, showing where a single instance of a detail texture can be painted by left clicking.

This tool is slightly buggy in the 1.10 tools. Dragging will paint all of the squares you drag the cursor across, but this ony works properly within the playable area of the map. Dragging outside may have unpredictable results, such as whole stripes of terrain being painted at once. This can be minimised by using single clicks to paint rather than dragging. If it behaves erratically within the play area, for instance by ceasing to paint, try changing the camera angle.

Alpha maps:

As mentioned in the previous section, Texture Stamping only shows through the grey and black areas of detail texture and decal alpha maps. Detail textures with no alpha transparency or a white alpha channel will completely cover any texture stamping you have done. Often, this is a source of confusion for people new to the ME: they lay a detail texture down that has no transparency, then try texture stamping and assume it isn't doing anything.

In the images below, you can see four different detail textures and the way they interact with the Texture Stamp:

Custom Detail Textures:

If you want to use your own texture art, it must be saved in DDS or TGA format at base2 resolutions (Typically 128*128, 256*256, and 512*512). An examination of texture formats and resolutions can be found in part 9 of these tutorials.

Any custom detail textures you make should be saved in [game directory]\WH40K\Data\Art\scenarios\textures\detail\

Once they are saved there, the ME will find them upon loading and add them to the tray list of detail textures. If the ME is already open they will not show up as you save them.

Different texture sizes effectively only change the resolution of the painted areas, not the size of the square terrain section that is painted. While appearing more detailed, a higher resolution image will cover the same amount of terrain as a lower resolution version, as shown below by a photograph of a piece of limestone at three different resolutions:

As mentioned in the ME basics section, the tiling of the detail textures can be changed in the Terrain Properties using the Detail Texture Repeat setting, as show below. The first image uses a repeat setting of 1, the second a repeat setting of 2:

The settings will go much higher, but at such levels will tend to scrub out any detail rather than enhance it in any way.

External links:

monoRAIL's detail textures tutorial
Making tiling textures in Photoshop
Gamasutra: The Power of the High Pass Filter