The Electroencephalogram (EEG) pattern of seizure activities is highly individual-dependent and requires experienced specialists to annotate seizure events. It is clinically time-consuming and error-prone to identify seizure activities by visually scanning EEG signals. Since EEG data are heavily under-represented, supervised learning techniques are not always practical, particularly when the data is not sufficiently labelled. Visualization of EEG data in low-dimensional feature space can ease the annotation to support subsequent supervised learning for seizure detection. Here, we leverage the benefit of both the time-frequency domain features and the Deep Boltzmann Machine (DBM) based unsupervised learning techniques to represent EEG signals in a 2-dimensional (2D) feature space. A novel unsupervised learning approach based on DBM, namely DBM_transient, is proposed by training DBM to a transient state for representing EEG signals in a 2D feature space and clustering seizure and non-seizure events visually. The effectiveness of DBM_transient is demonstrated on a widely-used benchmark dataset from Bonn University (Bonn dataset) and a raw clinical dataset from Chinese 301 Hospital (C301 dataset), with a large fisher discriminant value, surpassing the abilities of other dimensionality reduction methods, including DBM converged to an equilibrium state, Kernel Principal Component Analysis, Isometric Feature Mapping, t-distributed Stochastic Neighbour Embedding, Uniform Manifold Approximation. Such feature representation and visualization can help physicians to understand better the normal versus epileptic brain.