diff --git a/Thesis_Docs/Nikkhah_Nasab-Aida-Mastersthesis.pdf b/Thesis_Docs/Nikkhah_Nasab-Aida-Mastersthesis.pdf
index 7ebdeabc8c8b10b873e6ef27a053249b8a364180..3b9c13139e0167b1cb2d9701865c1e35687f15a7 100644
Binary files a/Thesis_Docs/Nikkhah_Nasab-Aida-Mastersthesis.pdf and b/Thesis_Docs/Nikkhah_Nasab-Aida-Mastersthesis.pdf differ
diff --git a/Thesis_Docs/main.tex b/Thesis_Docs/main.tex
index 5f2b405f24e8b5df021274f0149896ba00fe06e6..080f35f4ce7b42a51b04e8105107b3f696c14f18 100644
--- a/Thesis_Docs/main.tex
+++ b/Thesis_Docs/main.tex
@@ -606,7 +606,7 @@ By analyzing these patterns, organizations can gain valuable insights into user
\section{Distribution of Hosts Based on Unique URLs Contacted}
Understanding the interaction patterns of hosts within the network is for identifying key services, detecting anomalies, and optimizing network performance. This section analyzes the distribution of hosts based on the number of unique URLs they contacted, providing insights into the concentration of network activity.
-\begin{figure}[htbp]
+\begin{figure}
\centering
\includegraphics[width=\textwidth]{../Thesis_Docs/media/unic_urls.png}
\caption{Distribution of hosts based on unique URLs contacted. The X-axis represents the number of unique URLs contacted by each host, and the Y-axis shows the count of hosts in each category.}
@@ -649,68 +649,123 @@ After checking the URLs that were reached by these hosts, several conclusions ca
The data analysis presented in this chapter provides a comprehensive understanding of the dataset's structure, user behavior, and network interactions. By visualizing URL request counts, analyzing 24-hour visit patterns, examining time intervals between requests, and studying the distribution of hosts, this chapter uncovers key insights that can inform network optimization and security strategies. The findings highlight the importance of focusing on high-traffic URLs, understanding temporal patterns in user activity, and detecting periodic behavior that may indicate malicious beaconing. These insights lay the foundation for further analysis and the development of effective detection mechanisms in the BAYWATCH framework. By leveraging advanced visualization techniques and statistical methods, this chapter offers valuable insights into the dataset's characteristics and user behavior, providing a solid basis for enhancing network security and performance.
\chapter{Implementation}
-This chapter describes the novel extensions introduced in the DBAYWATCH framework. Although the original BAYWATCH framework was reimplemented in Python for better integration and maintainability, the focus here is on the additional contributions that extend beyond the original design. In particular, the work introduces an advanced signal analysis pipeline and presents a comprehensive evaluation using both real network traces and synthetically generated beaconing data. The enhancements center on refined temporal pattern detection through sophisticated filtering, frequency analysis, and autocorrelation techniques, leading to improved detection accuracy under varying jitter and interval conditions.
-\section{Enhancements Beyond BAYWATCH}
-The contributions extend the original beaconing detection methodology in several key aspects:
+This chapter details the novel extensions introduced in the BAYWATCH framework. Although the original BAYWATCH framework was reimplemented in Python for enhanced integration and maintainability, the focus here is strictly on the additional contributions that extend beyond the original design. The BAYWATCH framework incorporates an advanced signal analysis pipeline and a comprehensive evaluation methodology using both real network traces and synthetically generated beaconing data. These enhancements are aimed at improving temporal pattern detection under varied jitter, interval, and frequency conditions.
+
+\section{System Enhancements}
+The framework extends the original beaconing detection methodology with several keys:
+
+\subsection{Advanced Signal Analysis Pipeline}
+A multi-stage processing pipeline has been developed to isolate genuine periodic signals from noise. The pipeline consists of the following stages:
+
+\begin{enumerate}
+ \item \textbf{Bandpass Filtering:} \\
+ The initial stage involves preprocessing the raw time-series data to remove extraneous noise while preserving the essential periodic components. A zero-phase bandpass filter is applied to the data, ensuring that unwanted frequency components are suppressed without introducing phase distortions. This filtering step is critical as it isolates the frequency range where the periodic signals are expected to reside, thereby laying the foundation for subsequent analysis.
+
+ \item \textbf{Permutation-Based FFT Thresholding:} \\
+ Following filtering, the signal is transformed into the frequency domain using a Fast Fourier Transform (FFT). In order to distinguish significant periodic components from random noise, a dynamic threshold is computed. This threshold is derived by repeatedly randomizing the filtered data and analyzing the resulting spectral amplitudes. The underlying idea is that random permutations will destroy any inherent periodicity; therefore, frequency components in the original signal that exceed the threshold—determined based on a high confidence level—are likely to represent true periodic behavior.
+
+ \item \textbf{Autocorrelation Peak Detection:} \\
+ In parallel with the FFT analysis, the pipeline examines the time-domain characteristics of the filtered signal by computing its autocorrelation. This process highlights repeating patterns over time, with prominent peaks indicating potential periodicities. By identifying these peaks, the method is able to capture the candidate time lags at which the signal exhibits a high degree of self-similarity, reinforcing the evidence of an underlying periodic structure.
+
+ \item \textbf{Frequency-Lag Correlation:} \\
+ The final stage of the pipeline cross-validates the findings from the frequency and time domains. Here, the candidate frequency components obtained from the FFT are correlated with the candidate lags identified through the autocorrelation analysis. A tolerance is applied to account for minor discrepancies. Only those frequency components that consistently align with the time-domain peaks are retained as genuine periodic signals. This correlation ensures that the periodicity detected in the frequency spectrum is supported by corresponding temporal patterns, thereby enhancing the overall reliability of the detection process.
+\end{enumerate}
+
+This multi-stage pipeline represents a significant advancement in the detection of beaconing behavior, as it combines complementary analytical techniques to overcome the challenges posed by noisy and irregular data. The integration of bandpass filtering, dynamic FFT thresholding, and autocorrelation-based validation provides a robust framework for isolating true periodic signals, even in complex network traffic environments.
+
+This comprehensive signal analysis pipeline, combining zero-phase bandpass filtering, permutation-based FFT thresholding, autocorrelation peak detection, and frequency-lag correlation, constitutes a novel contribution of the DBAYWATCH framework. It is designed to be robust in the presence of noise and adaptable to variations in beaconing patterns, thereby significantly enhancing the detection of periodic signals in complex network traffic data.
+
+\subsection{Evaluation with Beaconing Data}
+A thorough evaluation was conducted using two data sources:
\begin{itemize}
- \item \textbf{Advanced Signal Analysis Pipeline:} A multi-stage processing pipeline was developed to refine beacon detection. The pipeline includes:
- \begin{enumerate}
- \item \textbf{Zero-Phase Bandpass Filtering:} Raw time-series data is cleansed of noise while preserving essential periodic features.
- \item \textbf{Permutation-Based FFT Thresholding:} A Fourier transform is applied to the filtered data, and a dynamic amplitude threshold is established by performing multiple random permutations. Only frequency components that exceed this threshold are considered significant.
- \item \textbf{Autocorrelation Peak Detection:} The autocorrelation function of the pre-filtered data is computed, and its prominent peaks are identified as candidate lags supporting the periodicity observed in the frequency domain.
- \item \textbf{Frequency-Lag Correlation:} Candidate frequencies from the FFT analysis are correlated with the autocorrelation peaks to ensure that only consistent periodic patterns are reported.
- \end{enumerate}
- \item \textbf{Extensive Evaluation with Beaconing Data:} A detailed evaluation was performed using both real network traces and synthetically generated beaconing data, enabling an assessment of detection performance under varied conditions.
+ \item \textbf{Real Network Traces:} Collected from an operational enterprise environment.
+ \item \textbf{Synthetic Beaconing Data:} Generated to emulate a wide range of conditions by varying the beacon interval and jitter.
\end{itemize}
-\section{Experimental Setup}
-In the synthetic experiments, a range of beacon configurations was simulated to study the impact of temporal noise on detection accuracy. For example, one configuration involved a beacon (beacon.example1.com) with a base interval of 300 seconds and a jitter of 2 seconds, meaning that the actual beacon intervals varied within a range of approximately 298 to 302 seconds. In another instance, a beacon (beacon.example2.com) was simulated with a 300-second interval and a jitter of 5 seconds, resulting in effective intervals between roughly 295 and 305 seconds.
+\section{Experimental Design of Synthetic Beaconing Data}
+The synthetic experiments simulated various beacon configurations to study the impact of temporal noise on detection accuracy. Each beacon candidate is characterized by:
+\begin{itemize}
+ \item \textbf{Base Interval (s):} The expected or nominal time gap between consecutive beacon transmissions, assuming no variations. For example, if a beacon transmits every 10 seconds, its base interval is 10 seconds.
+
+ \item \textbf{Jitter (s):} The amount of variation in the beacon’s transmission times. Instead of transmitting at fixed intervals, a beacon might send signals slightly earlier or later than expected. If the jitter is 5 seconds, the actual transmission times may vary within $\pm5$ seconds of the base interval, meaning a beacon scheduled every 10 seconds might actually transmit between 5 and 15 seconds.
+
+ \item \textbf{Jitter/Interval Ratio (\%):} This ratio quantifies the extent of timing variability relative to the base interval and is computed as:
+ \begin{equation}
+ \frac{\text{Jitter}}{\text{Interval}} \times 100\%.
+ \end{equation}
+ A lower ratio (e.g., 1--5\%) indicates minimal variation, making detection easier, whereas a higher ratio (e.g., 40--50\%) implies significant irregularity, complicating detection.
+
+ \item \textbf{Frequency (Hz):} The primary frequency of the beacon signal, representing how often a beacon transmits per second. It is calculated as the inverse of the average interval:
+ \begin{equation}
+ \text{Frequency} = \frac{1}{\text{Interval}}.
+ \end{equation}
+ For example, if a beacon transmits every 10 seconds, its frequency is:
+ \begin{equation}
+ \frac{1}{10} = 0.1 \text{ Hz}.
+ \end{equation}
+ A higher frequency corresponds to more frequent transmissions, facilitating detection.
+\end{itemize}
-Other configurations were designed with more significant perturbations. For instance, a beacon (beacon.example3.com) was configured with a 300-second base interval and a jitter of 120 seconds, leading to intervals ranging from 180 to 420 seconds, while yet another (beacon.example4.com) used a jitter of 150 seconds with the same base interval. Additionally, experiments included beacons with shorter intervals: one with a 10-second interval and a jitter of 5 seconds, another with a 10-second interval and a jitter of 2 seconds, as well as cases with 20-second intervals (with jitters of 5 and 10 seconds respectively), a 30-second interval with a 5-second jitter, a 40-second interval with a 5-second jitter, and a 50-second interval with a 5-second jitter. Finally, a beacon with a 120-second interval and a 10-second jitter was also evaluated. These diverse configurations allowed for the analysis of detection performance across a broad spectrum of periodicity regularity and noise levels.
+
+For instance:
+\begin{itemize}
+ \item A beacon with a 10-second interval and 2-second jitter has effective intervals between 8 and 12 seconds.
+ \item Another beacon with a 60-second interval and 10-second jitter produces intervals between approximately 50 and 70 seconds.
+ \item More extreme cases include a beacon with a 300-second interval and 150-second jitter (half of the interval), resulting in intervals from 150 to 450 seconds.
+\end{itemize}
\section{Results and Analysis}
-The performance of the detection algorithm was assessed using the amplitude score, which indicates the strength of the periodic signal, and by considering the effective jitter-to-interval ratio, which influences detection difficulty. Table~\ref{tab:beacon_comparison} summarizes the performance of the beacon candidates.
+Table~\ref{tab:beacon_comparison} presents an analysis of various beacon candidates. The table provides metrics that describe the periodicity of beacon transmissions, including interval duration, jitter, jitter-to-interval ratio, and frequency. These metrics are using for understanding the complexity of beacon detection and the impact of temporal noise on signal identification.
+
+This table provides an overview of beacon transmission characteristics, highlighting differences in periodicity and randomness. Beacons with high jitter introduce more unpredictability, making them harder to detect, while those with low jitter and frequent transmissions are easier to identify. The results underscore the importance of understanding temporal patterns in beacon signals and the impact of noise on detection accuracy.
\begin{table}
\centering
\caption{Beacon Candidates Ranked by Amplitude Score}
\label{tab:beacon_comparison}
- % Resize the table to fit the text width
\resizebox{\textwidth}{!}{%
- \begin{tabular}{lccccc}
+ \begin{tabular}{lcccc}
\toprule
\textbf{Beacon URL} &
\textbf{Interval (s)} &
\textbf{Jitter (s)} &
- \textbf{Jitter/Interval Ratio} &
- \textbf{Frequency (Hz)} &
- \textbf{Amplitude Score} \\
+ \textbf{Jitter/Interval Ratio (\%)} &
+ \textbf{Frequency (Hz)} \\
\midrule
- beacon.example1.com & 300 & 2 & 0.67\% & 0.0033 & 99.3\% (Very Easy) \\
- beacon.example2.com & 300 & 5 & 1.67\% & 0.0033 & 98.3\% (Easy) \\
- beacon.example3.com & 300 & 120 & 40\% & 0.0033 & 60\% (Very Hard) \\
- beacon.example4.com & 300 & 150 & 50\% & 0.0033 & 50\% (Extreme) \\
- beacon.example5.com & 10 & 5 & 50\% & 0.1 & 50\% (Very Hard) \\
- beacon.example6.com & 10 & 2 & 20\% & 0.1 & 80\% (Moderate-Hard) \\
- beacon.example7.com & 20 & 5 & 25\% & 0.05 & 75\% (Hard) \\
- beacon.example8.com & 20 & 10 & 50\% & 0.05 & 50\% (Very Hard) \\
- beacon.example9.com & 30 & 5 & 16.7\% & 0.033 & 83.3\% (Moderate) \\
- beacon.example10.com & 40 & 5 & 12.5\% & 0.025 & 87.5\% (Moderate) \\
- beacon.example11.com & 50 & 5 & 10\% & 0.02 & 90\% (Moderate) \\
- beacon.example12.com & 60 & 10 & 16.7\% & 0.0167 & 83.3\% (Moderate) \\
- beacon.example13.com & 120 & 10 & 8.3\% & 0.0083 & 91.7\% (Moderate) \\
+ beacon1.example.com & 300 & 2 & 0.67 & 0.0033 \\
+ beacon2.example.com & 300 & 5 & 1.67 & 0.0033 \\
+ beacon3.example.com & 300 & 120 & 40.00 & 0.0033 \\
+ beacon4.example.com & 300 & 150 & 50.00 & 0.0033 \\
+ beacon5.example.com & 10 & 5 & 50.00 & 0.1000 \\
+ beacon6.example.com & 10 & 2 & 20.00 & 0.1000 \\
+ beacon7.example.com & 20 & 5 & 25.00 & 0.0500 \\
+ beacon8.example.com & 20 & 10 & 50.00 & 0.0500 \\
+ beacon9.example.com & 30 & 5 & 16.67 & 0.0333 \\
+ beacon10.example.com & 40 & 5 & 12.50 & 0.0250 \\
+ beacon11.example.com & 50 & 5 & 10.00 & 0.0200 \\
+ beacon12.example.com & 60 & 10 & 16.70 & 0.0167 \\
+ beacon13.example.com & 120 & 10 & 8.30 & 0.0083 \\
\bottomrule
\end{tabular}%
}
\end{table}
-The analysis demonstrates that beacon candidates with low jitter relative to their intervals (e.g., beacon.example1.com and beacon.example2.com) produce strong periodic signals with high amplitude scores and minimal detection difficulty. In contrast, configurations with high jitter—resulting in jitter-to-interval ratios of 40--50\%—exhibit significantly reduced amplitude scores and increased detection difficulty. The experimental findings indicate that maintaining a jitter/interval ratio below approximately 10\% is optimal for robust periodicity detection.
+Figure \ref{fig:ABGraph} illustrates the synthetic beacon candidates with varying jitter levels. The x-axis represents the time intervals frequencies between beacon transmissions, while the y-axis shows the amplitude of the signals. The graph demonstrates how different jitter levels affect the periodicity and amplitude of beacon signals. Beacons with low jitter exhibit clear periodic patterns, making them easier to detect, while those with high jitter show more irregularity, complicating identification. By analyzing these synthetic beacon candidates, the framework can evaluate its detection capabilities under different noise conditions and refine its algorithms to improve accuracy and robustness. The figure presents the results obtained after applying all stages of the detection algorithm to the selected beacon URLs. Each URL exhibits distinct candidate points where periodic behavior has been detected. The analysis reveals a significant variation in the number of candidate points across different beacon URLs. Specifically, some beacons, such as "beacon1.example.com" through "beacon4.example.com", exhibit only a single candidate point, indicating that their periodic signals are either weak or occur over long intervals, making them more challenging to detect. This suggests that beaconing behaviors with longer intervals are inherently more difficult to detect, as their signals appear less frequently in the analyzed data. On the other hand, the algorithm performs more effectively in detecting beacons with shorter intervals. Other beacons, such as "beacon5.example.com" and "beacon6.example.com", show a substantially higher number of candidates, with 429 and 153 detected points, respectively, suggesting stronger periodicity and making them easier to identify.
+
+A clear example of this can be seen in "beacon7.example.com", where a detected frequency of 0.05 Hz corresponds to a periodic beaconing behavior every 20 seconds. The detection of this short-interval beacon illustrates the algorithm’s strength in identifying high-frequency periodic transmissions, as their repetition leads to more pronounced spectral features in the output. Overall, the figure demonstrates that while long-interval beacons pose detection challenges, the algorithm excels in identifying shorter-interval beacons with strong periodicity.
+
+\begin{figure}
+ \centering
+ \includegraphics[width=\textwidth]{../Thesis_Docs/media/artificialbeacons.png}
+ \caption{Synthetic Beacon Candidates with Varying Jitter Levels, the x-axis represents the time intervals frequencies between beacon transmissions, while the y-axis shows the amplitude of the signals.}
+ \label{fig:ABGraph}
+\end{figure}
-\section{Discussion and Conclusions}
-The enhancements introduced in the DBAYWATCH framework extend the capabilities of beacon detection beyond the original BAYWATCH implementation. The advanced signal analysis pipeline, incorporating refined filtering, permutation-based FFT thresholding, and autocorrelation peak detection, has demonstrated improved sensitivity and robustness in identifying periodic beaconing signals even in noisy conditions. The extensive evaluation with synthetic beacon data provided clear evidence that a lower jitter/interval ratio is critical for achieving a distinct and detectable periodic pattern.
+\section{Discussion and Conclusion}
+The BAYWATCH extensions significantly enhance beacon detection accuracy by incorporating an advanced signal analysis pipeline. The evaluation confirms that maintaining a low jitter/interval ratio (ideally below 10\%) is useful for robust detection. However, the framework also demonstrates resilience under moderate noise conditions through adaptive thresholding and correlation techniques. The synthetic experiments provide valuable insights into the impact of temporal noise on beacon detection, highlighting the importance of understanding periodic patterns and the challenges posed by irregular transmissions. By combining real-world network traces with synthetic beaconing data, the framework achieves a comprehensive evaluation, demonstrating its efficacy in detecting malicious beaconing behavior under varied conditions. These results underscore the framework's potential to enhance network security by identifying stealthy threats and improving anomaly detection capabilities. The advanced signal analysis pipeline, coupled with a rigorous evaluation methodology, positions the BAYWATCH framework as a valuable tool for securing enterprise networks against advanced cyber threats.
-In conclusion, the DBAYWATCH extensions not only enhance detection accuracy but also offer a scalable framework suitable for real-world network monitoring applications. The detailed experimental analysis has provided actionable insights into the optimal parameter configurations, paving the way for more robust and reliable network security solutions.
+The enhancements in the DBAYWATCH framework, as detailed in this chapter, offer substantial improvements over the original BAYWATCH implementation. By reimplementing the base framework in Python and extending it with an advanced signal analysis pipeline, DBAYWATCH achieves improved accuracy and scalability in beacon detection. The comprehensive evaluation with both real and synthetic data underscores the critical impact of jitter on detection performance and provides clear guidelines for optimal parameter settings in practical network security applications.
\chapter{Experiments and Discussions}
This chapter presents a comprehensive evaluation of the BAYWATCH framework to validate its efficacy in detecting malicious beaconing behavior in large-scale networks. The experiments are designed to address two objectives: first assessing the framework's robustness and accuracy under controlled noise conditions using synthetic datasets, and second evaluating its practical performance in real-world enterprise network environments. Synthetic data, generated with programmable noise levels and periodic patterns, enables systematic testing of BAYWATCH's core algorithms, such as the Fast Fourier Transform (FFT) and autocorrelation-based verification. Subsequently, the framework is deployed on a real-world dataset. This dual approach not only validates the theoretical soundness of the methodology but also demonstrates its scalability and operational feasibility. By synthesizing findings from both artificial and real-world scenarios, this chapter provides insights into BAYWATCH's strengths, limitations, and applicability in modern cybersecurity defense systems.
diff --git a/Thesis_Docs/media/artificialbeacons.png b/Thesis_Docs/media/artificialbeacons.png
new file mode 100644
index 0000000000000000000000000000000000000000..e38eb86bba6ddfca5e31261b34f5648f2eb684ad
Binary files /dev/null and b/Thesis_Docs/media/artificialbeacons.png differ