Maskless lithography (ML) systems have been researched as an alternative technologies of the conventional photolithography systems. Digital micromirror devices (DMD) can be used in ML systems as a role of photomask in the conventional photolithography systems. For high-throughput manufacturing processes DMDs in ML systems must be driven to their operational limits, often in harsh conditions. We propose an optical system and detection methodologies to detect DMD malfunctions to ensure perfect mask image transfer to the photoresist in ML systems. We categorize DMD malfunctions into two types. One is bad DMD pixel caused from mechanical defect and the other is data transfer error. We detect bad DMD pixels with 20×20 pixels using white and black image tests. We confirm data transfer errors at high frame rate operation of DMD by monitoring changes in the frame rate of a target DMD pixel driven by the input data with a set frame rate of up to 28,000 frames per second (fps).