1. There is an one-to-many relationship between track in input file and output file. You can create a profile that saves a track both as FLAC and DTS. Specifically, if you open a source with DTS-HD track, select "FLAC" profile and enable both DTS-HD tarck and DTS core, your output will have two audio tracks produced from the same source track. One would be DTS 5.1 copied as is and another would be DTS-HD track copied as FLAC. So you'll get a slightly bigger file but will keep the SPDIF-streamable DTS core.
2. MakeMKV uses channel order and channel bit mask as specified in Microsoft wavex specification. This is exactly the same channel order and channel mask that is used by libav/ffmpeg