diff --git a/chapters/2-Methodology.tex b/chapters/2-Methodology.tex index 0ad4332be2a3e7fa1f17fc414b0f0648f49fe78b..9e6bd5b6cd8c0d916b30b853552cb055dbfba88f 100644 --- a/chapters/2-Methodology.tex +++ b/chapters/2-Methodology.tex @@ -1 +1,10 @@ -\chapter{Methodology} \ No newline at end of file +\chapter{Methodology} + +This chapter outlines the methodology used to compare various image processing libraries. It describes the rationale behind the selected metrics, explains how these metrics are obtained and analyzed, and details the criteria used to select the libraries for evaluation. The approach is designed to yield objective, reproducible, and comprehensive performance comparisons, with a focus on two key metrics: image loading time and pixel iteration time. + +\input{sections/Chapter-2-sections/Overview.tex} +\input{sections/Chapter-2-sections/Definition.tex} +\input{sections/Chapter-2-sections/Measurement-Procedure.tex} +\input{sections/Chapter-2-sections/Data-Collection.tex} +\input{sections/Chapter-2-sections/Selection-Criteria.tex} +\input{sections/Chapter-2-sections/Summary.tex} diff --git a/sections/Chapter-2-sections/Data-Collection.tex b/sections/Chapter-2-sections/Data-Collection.tex new file mode 100644 index 0000000000000000000000000000000000000000..5861fc825ee9f1f6ba035e967886bca9f943e000 --- /dev/null +++ b/sections/Chapter-2-sections/Data-Collection.tex @@ -0,0 +1,33 @@ +\section{Data Collection and Analysis} + +To ensure that our performance measurements are both reliable and meaningful, we implement a rigorous data collection and analysis process: + +\subsection{ Repeated Trials and Averaging} + +Each experimental test (for both image loading and pixel iteration) is executed over a large number of iterations—typically 100 runs per library. This repetition helps smooth out transient variations and ensures that our measurements represent a consistent performance profile. The following steps are taken: + +\begin{itemize} + \item \textbf{Multiple Iterations:} For each library, the test is repeated numerous times under identical conditions. + \item \textbf{Statistical Averaging:} The mean time for each metric is computed from these iterations, providing a representative average performance figure. + \item \textbf{Variance Analysis:} Standard deviations and variances are calculated to assess the consistency of the results. Outlier values are identified and, if necessary, excluded to prevent skewed outcomes. +\end{itemize} + +\subsection{ Memory Profiling} + +In addition to timing metrics, we capture memory consumption data to understand the resource efficiency of each library. This involves: + +\begin{itemize} + \item \textbf{Memory Allocation Tracking:} Monitoring the amount of memory allocated during image loading and pixel iteration operations. + \item \textbf{Garbage Collection Monitoring:} Recording the frequency and duration of garbage collection events to assess their impact on performance. + \item \textbf{Profiling Tools:} Utilizing integrated tools such as BenchmarkDotNet’s memory profiler, along with additional system-level profiling utilities, we record the peak memory usage and total allocations. +\end{itemize} + +\subsection{ Statistical Analysis} + +The raw data collected from repeated trials is processed using statistical software to perform further analysis: + +\begin{itemize} + \item \textbf{Confidence Intervals:} Confidence intervals are calculated around the mean values to provide a measure of the reliability of our measurements. + \item \textbf{Comparative Statistical Tests:} Where applicable, we employ statistical tests (e.g., t-tests) to determine whether differences in performance metrics between libraries are statistically significant. + \item \textbf{Data Visualization:} Graphs and charts are generated to visually compare the performance distributions across libraries, offering an intuitive understanding of their relative efficiencies. +\end{itemize} \ No newline at end of file diff --git a/sections/Chapter-2-sections/Definition.tex b/sections/Chapter-2-sections/Definition.tex new file mode 100644 index 0000000000000000000000000000000000000000..639b629cd1cf8ad0fd76770e8fc40811302525f0 --- /dev/null +++ b/sections/Chapter-2-sections/Definition.tex @@ -0,0 +1,27 @@ +\section{Definition and Rationale for the Metrics} + +\subsection{ Image Loading Time} + +\textbf{Definition:} +Image loading time is defined as the total elapsed time required to retrieve an image from a file system, decode it into a standardized internal representation (e.g., RGBA32), and initialize any associated data structures needed for further processing. + +\textbf{Rationale:} +\begin{itemize} + \item \textbf{I/O and Decoding Efficiency:} The time taken to load an image is a direct indicator of how efficiently a library handles input/output operations and decodes various image formats. In industrial applications, where images may be read in real time from cameras or sensors, fast loading is essential. + \item \textbf{Baseline for Processing Pipelines:} Since loading is the first step in any image processing pipeline, delays at this stage can cascade and affect overall system performance. An efficient image loading mechanism can significantly reduce the latency of the entire workflow. + \item \textbf{Minimizing Overhead:} The loading process also involves memory allocation and data structure initialization. A library that minimizes overhead in these areas will allow more resources to be devoted to the actual image processing tasks. + \item \textbf{Standardized Measurement:} The loading process can be uniformly measured across different libraries by using the same image datasets and environmental conditions, ensuring that the comparisons are fair and reproducible. +\end{itemize} + +\subsection{ Pixel Iteration Time} + +\textbf{Definition:} +Pixel iteration time refers to the duration required to traverse every pixel in an image and apply a predefined, uniform operation—such as converting an image to grayscale. This metric captures the efficiency of the library’s core routines for low-level data manipulation. + +\textbf{Rationale:} +\begin{itemize} + \item \textbf{Core Computation Performance:} Many image processing tasks (e.g., filtering, thresholding, and color adjustments) require examining or modifying every pixel. The speed at which a library can iterate over pixels is a direct measure of its processing efficiency. + \item \textbf{Algorithmic Sensitivity:} Pixel-level operations are sensitive to implementation details like loop unrolling, memory access patterns, and caching strategies. Faster iteration times imply better-optimized routines. + \item \textbf{Simplicity and Reproducibility:} Converting an image to grayscale is a simple and commonly used operation that can serve as a proxy for other pixel-level tasks. Its simplicity makes it an ideal candidate for standardizing comparisons across libraries. + \item \textbf{Isolation of Low-Level Performance:} By isolating the pixel iteration operation from higher-level tasks, we can specifically evaluate the efficiency of the library’s data structures and internal algorithms without interference from more complex operations. +\end{itemize} \ No newline at end of file diff --git a/sections/Chapter-2-sections/Measurement-Procedure.tex b/sections/Chapter-2-sections/Measurement-Procedure.tex new file mode 100644 index 0000000000000000000000000000000000000000..e2de2991771cb0279059c4b8d141dd05c4751871 --- /dev/null +++ b/sections/Chapter-2-sections/Measurement-Procedure.tex @@ -0,0 +1,43 @@ +\section{Measurement Procedure} + +\subsection{ Experimental Setup} + +To ensure consistency and reliability, all tests are performed under controlled conditions: + +\begin{itemize} + \item \textbf{Hardware Consistency:} All experiments are conducted on the same machine with a fixed hardware configuration. This eliminates variability due to differences in CPU speed, memory, or storage performance. + \item \textbf{Software Environment:} We use a consistent operating system and development environment (e.g., .NET framework) across all tests. Timing is measured using high-precision tools such as BenchmarkDotNet to capture accurate performance metrics. + \item \textbf{Image Dataset:} A standardized dataset of images is used for testing. This dataset includes images of varying resolutions and formats to simulate real-world industrial scenarios. + \item \textbf{Repetition and Averaging:} Each test is repeated multiple times (e.g., 100 iterations) to account for random fluctuations and to ensure that the measured performance is statistically significant. The average and variance of the results are computed to assess consistency. +\end{itemize} + +\subsection{ Measuring Image Loading Time} + +The procedure for measuring image loading time consists of the following steps: + +\begin{itemize} + \item \textbf{File Access Initiation:} The test begins by initiating a file read operation from the disk. The image file is selected from a predetermined dataset. + \item \textbf{Decoding and Conversion:} Once the file is accessed, the image is decoded into a standardized internal format, such as RGBA32. This step includes converting the raw image data (e.g., JPEG, PNG) into a format that is readily usable by the library. + \item \textbf{Initialization of Data Structures:} Any necessary memory allocation and initialization of internal data structures are performed at this stage. + \item \textbf{Timing the Operation:} A high-resolution timer records the time from the initiation of the file read operation until the image is fully loaded and ready for processing. + \item \textbf{Repetition and Averaging:} This process is repeated multiple times, and the average loading time is computed. Variability in the measurements is analyzed using standard deviation metrics to ensure reproducibility. +\end{itemize} + +\subsection{ Measuring Pixel Iteration Time} + +The measurement of pixel iteration time is carried out in a similar systematic manner: +\begin{itemize} + \item \textbf{Image Loading:} Prior to the iteration test, the image is loaded into memory using the same process as described above. This ensures that the image is in a known, consistent state. + \item \textbf{Pixel Operation Execution:} A simple operation is defined (e.g., converting each pixel to its grayscale equivalent). The algorithm iterates over every pixel, reading its RGB values and computing the grayscale value based on a weighted sum. + \item \textbf{Timing the Iteration:} The entire pixel iteration process is timed from the moment the iteration begins until every pixel has been processed. High-precision timers are used to capture this duration. + \item \textbf{Isolation of the Operation:} To ensure that the measurement reflects only the time for pixel iteration, other processes (such as file I/O) are not included in this timing. + \item \textbf{Multiple Iterations:} Like the image loading test, the pixel iteration test is repeated many times (e.g., 100 iterations) to obtain an average processing time. Outliers are analyzed and removed if they are deemed to be due to external interference. +\end{itemize} + +\subsection{ Tools and Instrumentation} + +\begin{itemize} + \item \textbf{Benchmarking Framework:} BenchmarkDotNet is used as the primary tool for performance measurement. It provides accurate timing measurements and can also track memory usage. + \item \textbf{Profiling Utilities:} Additional profiling tools are employed to monitor memory allocation and garbage collection events. This ensures that both time and resource consumption are captured. + \item \textbf{Data Logging:} All measurements are logged for further statistical analysis. This raw data is later processed to compute averages, standard deviations, and confidence intervals, forming the basis for our comparative analysis. +\end{itemize} \ No newline at end of file diff --git a/sections/Chapter-2-sections/Overview.tex b/sections/Chapter-2-sections/Overview.tex new file mode 100644 index 0000000000000000000000000000000000000000..e3092c7ed92224e0b6110673be2138abd26c8333 --- /dev/null +++ b/sections/Chapter-2-sections/Overview.tex @@ -0,0 +1,19 @@ +\section{Overview} + +Image processing tasks in industrial applications typically involve two fundamental operations: acquiring image data (loading) and performing pixel-level computations (iteration). The efficiency of these operations directly influences the overall performance of any image processing system. In our evaluation, we have chosen to focus on two key metrics: + +\begin{itemize} + \item \textbf{Image Loading Time:} The time taken to load an image from persistent storage into the system's memory. + \item \textbf{Pixel Iteration Time:} The duration required to traverse and process each pixel in an image, exemplified by converting the image to grayscale. +\end{itemize} + +These metrics are chosen for several reasons: + +\begin{itemize} + \item \textbf{Universality:} Both operations are common to nearly all image processing workflows, regardless of the complexity of the subsequent processing steps. + \item \textbf{Fundamental Performance Indicators:} The speed of image loading reflects the efficiency of file I/O, image decoding, and memory allocation, while pixel iteration performance indicates how well a library can handle low-level data manipulation—an operation that is central to filtering, enhancement, and other pixel-based computations. + \item \textbf{Comparability and Reproducibility:} By using standardized tasks that all libraries must perform, we can compare their performance on a like-for-like basis. This approach minimizes variability and provides a clear baseline for comparing otherwise diverse systems. + \item \textbf{Hardware Independence:} These metrics are less influenced by high-level algorithmic choices and more by the underlying implementation and optimizations, making them suitable for benchmarking across different libraries and platforms. +\end{itemize} + +While there are many other potential metrics (such as encoding time, advanced filtering speed, or transformation accuracy), we selected image loading and pixel iteration because they are both critical and universally applicable operations. They provide a controlled environment for performance measurement and are directly relevant to the low-level efficiency needed in industrial scenarios. \ No newline at end of file diff --git a/sections/Chapter-2-sections/Selection-Criteria.tex b/sections/Chapter-2-sections/Selection-Criteria.tex new file mode 100644 index 0000000000000000000000000000000000000000..64c60729b67711ef5cb15ffd8f713bee06dfa6be --- /dev/null +++ b/sections/Chapter-2-sections/Selection-Criteria.tex @@ -0,0 +1,52 @@ +\section{Selection Criteria for Image Processing Libraries} + +Selecting the appropriate libraries for our comparison is a critical step that shapes the overall evaluation. We employ a set of comprehensive criteria to ensure that only relevant and robust image processing libraries are included: + +\subsection{ Functional Coverage} + +The primary requirement is that the library must support the core operations fundamental to image processing: + +\begin{itemize} + \item \textbf{Image Loading and Creation:} The library should efficiently load images from various formats and support the creation of new images. + \item \textbf{Pixel Manipulation:} It must provide mechanisms for direct pixel access and manipulation, which are essential for tasks like filtering, transformation, and color adjustments. + \item \textbf{Transformation Capabilities:} Support for resizing, cropping, and color space conversions is essential to evaluate overall processing flexibility. +\end{itemize} + +\subsection{ Performance and Resource Efficiency} + +Given the industrial context, the following performance aspects are prioritized: + +\begin{itemize} + \item \textbf{Low Image Loading Time:} Efficient I/O and decoding capabilities ensure that the system can handle high volumes of image data. + \item \textbf{Fast Pixel Iteration:} The library must exhibit optimized routines for traversing and processing pixels, indicating low-level efficiency. + \item \textbf{Memory Usage:} Efficient memory management is critical, particularly when processing high-resolution images or large batches. Libraries with minimal memory overhead and low garbage collection impact are preferred. +\end{itemize} + +\subsection{ Ease of Integration} + +Practical integration into existing industrial systems is another important criterion: + +\begin{itemize} + \item \textbf{System Compatibility:} The library should seamlessly integrate with our existing software stack (e.g., the .NET framework). + \item \textbf{Documentation and Community Support:} Comprehensive documentation and active community support facilitate adoption and troubleshooting. + \item \textbf{Modularity and Extensibility:} A modular design that allows for the easy addition of custom functionalities is advantageous for industrial applications that may have evolving requirements. +\end{itemize} + +\subsection{ Licensing and Cost Considerations} + +While the focus is on performance, practical deployment also depends on the licensing terms and cost: + +\begin{itemize} + \item \textbf{Open-Source or Cost-Effective Licensing:} Libraries with permissive or cost-effective licenses are preferred, as they reduce the total cost of ownership. + \item \textbf{Long-Term Maintenance:} Consideration of ongoing maintenance costs and the ease of future updates is essential for sustainable industrial deployment. +\end{itemize} + +\subsection{ Relevance to Industrial Applications} + +Finally, the chosen libraries must demonstrate applicability to real-world industrial scenarios: + +\begin{itemize} + \item \textbf{Real-Time Processing:} The ability to handle real-time image processing is crucial for applications such as quality control and automated inspection. + \item \textbf{Scalability:} The library should efficiently manage large datasets and high-resolution images. + \item \textbf{Robustness:} Proven reliability in diverse industrial conditions (e.g., varying lighting, environmental noise) is essential for practical deployment. +\end{itemize} \ No newline at end of file diff --git a/sections/Chapter-2-sections/Summary.tex b/sections/Chapter-2-sections/Summary.tex new file mode 100644 index 0000000000000000000000000000000000000000..901666e3c7905b89452c4174793e8c9398218286 --- /dev/null +++ b/sections/Chapter-2-sections/Summary.tex @@ -0,0 +1,9 @@ +\section{Summary} + +This chapter has presented a detailed methodology for the comparative evaluation of image processing libraries. We focused on two fundamental performance metrics—image loading time and pixel iteration time—selected for their universality, reproducibility, and direct relevance to low-level image processing tasks. The measurement procedures involve controlled experiments using standardized image datasets, with repeated trials to ensure statistical reliability and comprehensive memory profiling to capture resource efficiency. + +Furthermore, we outlined the selection criteria used to choose the libraries for evaluation, which include functional coverage, performance, ease of integration, licensing, and industrial applicability. By applying these criteria, we ensure that our comparative analysis is grounded in practical relevance and technical rigor. + +The methodology described in this chapter forms the backbone of our experimental evaluation. It provides a clear, structured framework for measuring and analyzing the performance of different image processing libraries. This framework not only facilitates direct comparisons but also helps identify trade-offs between speed, memory efficiency, and ease of integration—factors that are critical for the deployment of image processing solutions in industrial applications. + +With the methodology established, the following chapters will present the experimental results and discuss their implications for selecting the most suitable image processing library for industrial applications. \ No newline at end of file