Web-based Application for Cancerous Object Segmentation in Ultrasound Images Using Active Contour Method

Segmentation, or the process of separating clinical objects from surrounding tissue in medical images, is an important step in the Computer-Aided Diagnosis (CAD) system. The CAD system is developed to assist radiologists in diagnosing cancer malignancy, which in this research is found in ultrasound (US) medical imaging. The manual segmentation process, which cannot be accessed remotely


Introduction
The stages in a CAD (Computer Aided Diagnosis) system are preprocessing, segmentation, feature extraction, and classification (Nugroho, Hidayat, Adi Nugroho, et al. 2020).Segmentation plays a crucial role in CAD systems (Noble 2010), as it is used to separate clinical objects from surrounding tissues in medical images.The segmentation stage is critical because it provides objective data such as geometric information, shape, edge boundaries, and textures that are useful for subsequent steps, such as feature extraction and classification, to obtain accurate diagnostic results (Nugroho et al. 2019).
CAD systems are developed to assist radiologists in diagnosing cancer malignancy, specifically in this research, using medical imaging techniques such as ultrasonography (US).The US is used as a screening tool for suspicious objects such as breast lesions and thyroid nodules in cancer diseases (Nugroho et al. 2021a).CAD systems are used as a second opinion by radiologists due to the subjective nature of ultrasound (US) readings, which leads to high variability in the diagnostic process (Rodríguez-Cristerna et al. 2017) caused by the poor quality of US images (Munarto et al. 2018).Cancer-related mortality rates in Indonesia are indeed quite high.According to Globocan 2020 data, the prevalence of cancer deaths in Indonesia over the past 5 years reached 946,088 cases (Globocan 2020).The RS80 is not only used to detect thyroid cancer objects but also breast lesions (Electrónica Y Medicina SA 2015).
However, the machine still has a weakness in the segmentation stage.In this stage, to perform segmentation, radiologists still need to manually initialize the location of the cancer object as seen in Figure 1 point b, and need to select the desired segmentation results as shown in Figure 1 point c.In addition, the RS80 machine is not supported with easy access that allows radiologists to diagnose stored data (repository) without direct contact with the ultrasound machine.The era of big data allows for storing many ultrasound images that accumulate due to many routine patient check-ups each day.The advanced technology of the US needs to be accompanied by ease of access for radiologists and clinical operators.
The manual segmentation process, which cannot be accessed remotely, poses a limitation for CAD systems.This is because screening for cancer objects is a repetitive and continuous task that needs to be performed frequently.If done manually, it would be ineffective and prone to human errors due to the exhaustive nature of the work.
To enhance the effectiveness of CAD systems in medical imaging diagnosis, several research studies have been conducted.However, these studies have mainly focused on developing the formulation methods rather than implementing them in a user-friendly interface that can be directly used by medical professionals.Almajalid et al. (2018) developed a segmentation method based on the U-Net deep learning architecture for breast ultrasound imaging to improve the accuracy of diagnosis in a CAD system (Almajalid et al. 2018).The research findings demonstrated that the modified U-Net method was more robust and accurate in segmenting breast lesions in ultrasound images, outperforming the other two methods significantly with a Dice coefficient (DC) of 0.825 and a similarity level of 0.698.Nugroho et al. (2020) conducted a study by developing a method called combinatorial active contour bilateral filter (Nugroho, Hidayat, Adi Nugroho, et al. 2020).The research presented a combined framework for ultrasound image segmentation using a bilateral filter (BF) and Region-Edge-based AC model.The research findings showed a Dice coefficient value of 90.05 ± 5.81% using a dataset consisting of 258 ultrasound images with corresponding ground truth.
The research conducted by Pathan et al. (2022) implemented an artificial intelligence-based method for rapid breast cancer detection in ultrasound images into a web application (Pathan et al. 2022).The research findings demonstrated that the proposed framework was effective, achieving a test accuracy of 81.02%.Therefore, it can be used in CAD systems to reduce human errors in determining diagnostic outcomes.
In 2023, Nugroho 2023 developed a web application that adopts the method called MoRbAC (Morphological Region-Based Active Contour) for automatically detecting cancer objects in ultrasound images (Nugroho et al. 2023).MoRbAC is a combinatorial framework consisting of a simplified Chan-Vese (CV) active contour model and morphological image operations.The simplified CV serves to segment the objects, while the morphological operations play a role in reducing detection errors and localizing them specifically (Nugroho 2022).The proposed web-based application has been validated on 20 breast lesion and thyroid nodule ultrasound images.Python programming with the Flask framework was used to deploy this application.The validated results are compatible with various browsers and achieve an average accuracy of 98.75%.This achievement demonstrates that MoRbAC is suitable for use as an ultrasound CAD detection technique in a web service system.
Based on the literature, most CAD developments primarily consist of formal models and program algorithms.The development of CAD systems aimed at practical usability, such as web-based CAD, especially focusing on the segmentation stage, is still relatively limited.So, how can we build an application that provides easy access for radiologists to segment cancer objects in ultrasound images within an automated CAD system?
This research aims to address that question by developing a web-based application for CAD that adopts the active contour (AC) method (Nugroho et al. 2021b).The AC method has the potential for automated segmentation.A web-based application was chosen due to its various advantages, such as accessibility at any time and from anywhere if the device is connected to the internet.This allows radiologists to perform segmentation processes more efficiently and in a user-friendly manner.
In this research, the developer will adopt the AC method to be implemented in a web application that can be used by medical professionals to perform cancer object segmentation on ultrasound images.The web application is expected to contribute to the creation of a more modern technology that can be utilized as a supportive system in the process of effectively and efficiently diagnosing cancer malignancy in ultrasound images.Modern health technology has the potential to bring about various advantages, including cost reduction in healthcare and prevention of potential health issues (Chrisdianti et al. 2023).
In addition, by developing web applications in this study, radiologists also gain benefits such as implementing a segmentation process that is more flexible and easier to perform with accurate segmentation results from the object being investigated.
The structure of this paper is as follows: after explaining the background, research gaps, and study objectives in the Introduction, a review and theory used in Literature Reviews.Next, the development of the web application will be discussed in the Methodology.Results and Discussion will explain the research findings, including the various predetermined testing schemes.Lastly, the Conclusion will summarize the research findings and provide recommendations for further studies.

Similar Studies
Segmentation is an effort to partition an image into desired regions through the separation of information required from data for further purposes or objectives (Chen et

Active Contour (AC)
The effectiveness of the AC method is demonstrated by several studies above, the AC method is chosen in this study to perform the segmentation of cancer objects on ultrasound images and implemented in a web-based application.Active contour (AC) is a model that is highly effective in medical image processing, as it can separate the necessary pixels from the foreground (Kass et al. 1988).The main application of AC in image processing is to define smooth shapes in the image and form a closed contour for the region.The AC method is divided into three (3) models, local edge-based, region-based, and hybrid model.
Local edge-based methods like GAC (Geodesic Active Contour) (Caselles et al. 1997) are capable of performing specific object segmentation but lack detection capabilities.On the other hand, global region-based methods like CV (Chan-Vese) (Chan and Vese 2001) can perform segmentation on the entire image but lack specificity.Therefore, there is a need for an automated method that combines the strengths of both approaches and transitions seamlessly between global and local segmentation.In this research, the HERBAC (Hybrid Edge and Region-Based Active Contour) model (Nugroho et al. 2021b) is adopted and implemented in a web application.

HERBAC Method
The HERBAC (Hybrid Edge and Region-based Active Contour) method combines the CV model for global segmentation and GAC for accurate local segmentation.This automated method is applied for object detection to generate initial contour estimates using the simplified formulation of CV (Nugroho et al. 2021b).To achieve automatic detection of cancerous objects, the CV formulation has been simplified in a previous study (Nugroho 2022).The simplified regularized CV model, followed by morphological operations, is utilized for detecting cancerous objects in ultrasound images.The proposed method demonstrates relatively high performance, with an average IoU score of 84.20% obtained from validation tests using 20 thyroid and breast ultrasound images.
The HERBAC method was chosen because it has been tested for segmenting 114 breast and thyroid radiology ultrasound images, achieving a high accuracy of 92.31%.These results demonstrate the effectiveness and efficiency of the proposed method, indicating its potential for practical implementation in CAD systems for radiology ultrasound imaging.

Methodology
Several methods for developing web applications have been created, such as waterfall, agile, rapid prototyping, spiral, incremental, and others.However, in this research, the waterfall method was chosen as the approach for developing the web application.The stages of development are shown in Figure 2.
The waterfall method was chosen over other methods in this research because of its sequential and nonoverlapping stages, which progress from top to bottom (Balaji and Murugaiyan 2012).Each stage has a designated timeframe and is completed before moving on to the next stage.This methodology enables effective project deadline management.Additionally, the waterfall model allows for the identification and resolution of design flaws in the system before final product development, as the requirements are well-defined before the commencement of development.Compared to other models like agile, which is developed iteratively to accommodate changes in a project, the waterfall model is more suitable for well-conceptualized projects that do not require continuous changes, as in this research.Additionally, the waterfall model has been used by several studies in developing web applications (Alsagaby and Alharbi 2021;Maulana et al. 2021).
The research was conducted at the E6 Building Research Laboratory, Faculty of Engineering, UNNES.
It involved the construction of the HERBAC method algorithm programming using the Python programming language, which took approximately eight months (from March to October 2022).This was followed by the development of a Python-based web application using the Django framework, which served as the intermediary for user interaction with the application for a period of four months (from November 2022 to February 2023).
The tools used in designing the web application in this research include hardware specifications of a desktop computer with an Intel (R) Core (TM) i7-2600 CPU @ 3.40GHz processor, 4.00 GB of RAM, and an OS of Windows 7 64-bit.The software used includes the Python programming language, Spyder (Text Editor), Visual Studio Code (Text Editor), and Google Chrome (browser).Additionally, the materials used in this study include a dataset of 114 thyroid and breast cancer images obtained from the Department of Radiology, RSUP Dr. Sardjito and Dr. S. Hardjolukito, The Air Force Central Hospital in Yogyakarta (Nugroho et al. 2021b;Nugroho, Hidayat, Nugroho, et al. 2020).
This research involved several parties, including radiologists as experts to test the developed web application, practitioners knowledgeable in the HERBAC method to test the algorithm's suitability on the web, and common people as direct users to evaluate the effectiveness of the web application before it was tested by experts.The analysis and requirements specification phase aims to determine the resources needed for the development of the web application, including functional and non-functional requirements.In line with the research objectives, the developed web application is intended to be user-friendly, easily accessible, and assist radiologists in diagnosing cancer malignancy by effectively implementing the adopted HERBAC algorithm.Therefore, the web application is designed to be simple and easily understandable, even for common people.Based on this and consultations with experts, the developer analyzed the functional and non-functional requirements of the web application, as shown in the following Table 1.Web design is used for preparation before implementation by writing code to facilitate web development based on the results of the analysis conducted.
Next, the coding stage follows.This stage involves implementing the HERBAC algorithm into a code program.The HERBAC algorithm is implemented in the Python programming language.Python is an easy-to-use and open-source programming language (Vallat 2018).Python also has many advanced libraries such as OpenCV and Scikit-Image for image processing, Numpy for scientific calculations, and Django as a web framework.These four libraries are the main libraries used in this research to develop the web application.The coding process then continues with the creation of the user interface.This step is divided into two parts, coding for the backend and frontend.The backend is done by calling the functions that have been created in the previous step to be displayed in the browser using the Django framework.Next, the front-end stage is done to create templates using HTML and CSS languages according to the designed user interface.
The next step is testing.The web application will be tested using several criteria in the ISO 25010 model, which include functionality aspect using black box testing, portability by simulating web usage on various devices, efficiency using GTmetrix tools, reliability using WAPT tools, as well as expert validation and usability testing using a questionnaire to experts.
1.The functionality testing is conducted to check whether the web application complies with the intended specifications and functional requirements created in the requirements analysis.This testing uses the black box testing method.Black box testing is a testing method that focuses on the functional specification of software with testers defining a set of input conditions and testing the functional specification of the program (Hidayat and Muttaqin 2018).After the functionality testing, the results will be obtained in the form of the number of valid and invalid functions.To analyze the data, standard interpretations specified in ISO/IEC TR 9126-2:2002 will be used (Nalarita and Listiawan 2018).The formula for data analysis used is shown in Equation 1. the security of the web application and stored data is ensured after the US image is uploaded, the web performs a segmentation process using the HERBAC method the web displays output in the form of three images: the first image is the input image, the second is the detected image, and the third is the segmented image using the HERBAC method, and there is an information section to find out the name of the uploaded image, area, etc.
users can see the segmentation process after the output is displayed with a button called process which is displayed in gif format users can delete input and output images by clicking the Delete button, which then redirects the user to the Segmentation page users can carry out the segmentation process again by re-uploading the desired ultrasound image users can find information about the development of the segmentation method used on the about page by clicking on the navbar 4. Reliability testing is done by putting a certain load on the web to determine whether the web can run well or not.In this study, reliability testing used the WAPT (Web Application Load, Stress, and Performance Testing) software tool that will perform stress testing through active user simulation to obtain the percentage of success in terms of sessions, pages, and hits (Ma'arief et al. 2019).The results of stress testing must meet Telcordia standards, which are at least 95% (Asthana and Olivieri 2009). 5.The next step is expert validation testing.This test is carried out to validate the suitability of the HERBAC algorithm in the web application with experts using a questionnaire method.The experts are the main authors of the HERBAC algorithm in the reference article entitled "Development of Active Contour Model for Radiological Ultrasound Image Segmentation" (Nugroho et al. 2021b).
The data analysis technique used to measure the feasibility of the web application in the expert validation test is the percentage formula of application feasibility according to (Sugiyono 2015) as shown in Equation 3.
Where:  = Application feasibility percentage ∑  = Total score of the evaluation aspects by the experts ∑  = The maximum score of the assessment 6. Usability testing is used to determine the feasibility of a web application.This usability testing uses a questionnaire method for radiologists.From each questionnaire result, data will be processed into a conclusion of whether the built application is feasible or not.There are five indicators in determining the questionnaire questions in usability testing.Learnability which is related to the user's ease of operating the web application.Efficiency which explains the user's speed level in using and learning the web application.Satisfaction which explains the user's satisfaction level in using the web application.The error explains that the web application is consistent when used by users so that it does not experience problems.Memorability is the ability to remember and the user's ease without having to learn again after not using it for a long time (Setiawan and Widyanto 2018).
The data analysis technique used to measure the feasibility of the web application in usability testing is to use the formula in Equation 3.
At the final stage of the waterfall method, which is the deployment phase, the application that has passed the tests is distributed to the users.In phase, deployment is done by hosting the website to publish it so that users can access it on the internet.

Results and Discussion
The result of this study is the development of a web application named Cosion (Cancerous Object Segmentation) using the waterfall model.Cosion can be accessed through http://cosion.my.id/.Cosion is responsive, as shown in Figure 3, and has three main pages, Home, Segmentation, and About.The Home page contains brief information about Cosion, and there is a Segmentation menu that users can select to perform the segmentation process as shown in Figure 4. Furthermore, the Segmentation page starts with an Upload file page as shown in Figure 5, where users can upload a US image.This is followed by displaying the result as shown in Figure 6 if the segmentation process has been completed, which contains the input and output images of the segmentation result.The About page, as shown in Figure 7, contains information about Cosion, along with menus that users can select for further information about the segmentation method used.In addition, there is also information about the libraries used in web development packaged in the library logo image.

Functionality Testing
The testing of Cosion on the aspect of functionality using black box testing resulted in the web application being rated as good.The tested features include the menu on the navbar, the home page, segmentation features that encompass the upload process and segmentation until generating the output, as well as the features on the About page.
Each feature within Cosion can function properly as expected.This is evidenced by the X value generated which is close to 1, specifically 0.9375 as shown in Table 2.A good functionality value is achieved if X approaches 1 according to ISO/IEC TR 9126-2:2002 measurement standards, which is 0 ≤ X ≤ 1.

Portability Testing
According to the results of the portability testing as shown in Table 3, the Cosion web application that was built can be accessed well on various browsers in different OS and devices, both on desktop and mobile if there is an internet connection, except an error when displaying the menu bars in responsive mode on the Safari browser.However, this error does not affect the segmentation process.The segmentation results are still displayed properly and can be accessed through alternative menus available on each page.

Efficiency Testing
The efficiency testing results using GTmetrix by testing all pages on the Cosion web application revealed that it was rated as good and can be further developed, as shown in Table 4 which categorizes Cosion as Grade A with a score of 96.43±0.66%and an average time required to load a page on Cosion of 1.8±0.36seconds.

Reliability Testing
The reliability testing results using WAPT to find the percentage of success in terms of sessions, pages, and hits with stress testing simulation shown in Table 5.Both prove that Cosion web application can perform well when given a load according to the scenario, which is a load of 20 active users within 5 minutes, with 2 users logging in every 10 seconds.The stress testing percentage obtained is 100%.This result has met the Telcordia standard, which is a minimum of 95%.

Experts Method Validation and Usability Testing
Questionnaire testing was conducted with experts in both the HERBAC method validation and usability testing.Both tests used a Likert scale of 1 to 4. A value of 1 means Not Suitable or Poor, 2 means Less Suitable or Not Good, 3 indicates Suitable or Good, and 4 means Highly Suitable or Very Good.The results were then processed using Equation 3 and will be categorized as Not Suitable, Less Suitable, Suitable, or Highly Suitable.The test instruments for expert validation testing of the HERBAC method are shown in Table 6.The validation testing of the HERBAC method to determine the accuracy of the HERBAC method in the Cosion web application obtained a percentage of 77.5%.This result proves that all features in the HERBAC method have been adopted in the Cosion web application, and the HERBAC workflow has been executed the same in the web application.
According to Table 7, the usability testing using a questionnaire aimed at medical experts, specifically three specialist doctors, resulted in a percentage of 75.8%.The questionnaire consisted of 16 questions divided into 5 indicators, including Learnability (5 questions with a maximum score of 20), Efficiency (3 questions with a maximum score of 12), Satisfaction (5 questions with a maximum score of 20), Error (2 questions with a maximum score of 8), and Memorability (1 question with a maximum score of 4).Both percentages indicate that the Cosion web application is suitable for use by radiologists in assisting the cancer malignancy diagnosis process in the segmentation stage.

Discussion
This research developed a computerized CAD system based on a website to assist radiologists in segmenting suspected cancer objects during the cancer malignancy diagnosis process.To perform the segmentation process, this research adopted an active contour method called HERBAC.To assess the suitability of the HERBAC algorithm implemented in the Cosion web application, a validation test was conducted with the method's creator.The test resulted in a percentage of 77.5%, indicating that Cosion is deemed suitable for adopting the HERBAC algorithm.
The Cosion web application is considered good according to testing on the functionality aspect using black box testing with an X value of 0.9375.Each feature in the Cosion web application can function properly as expected.The Cosion web application can be accessed well in various browsers on various operating systems and devices, both desktop and mobile if there is an internet connection.All pages in the Cosion web application are rated good and can be developed according to the efficiency testing results using GTmetrix with a score of 96.43±0.66% in the Grade A category and an average load time of 1.8±0.36seconds per page.The Cosion web application can run well in terms of reliability with stress testing simulations and meets Telcordia standards when given a load according to the scenario, which is a load of 20 active users for 5 minutes with 2 users entering every 10 seconds.The stress testing percentage obtained was 100%.In usability testing, Cosion was tested on radiologists with a resulting percentage of 74.45%.
Based on the usability testing results, Cosion can be considered learnable.It was deemed relatively easy to understand and the presented information and content were clear and specific.In terms of efficiency, the results indicate that Cosion is not yet able to perform segmentation quickly.However, all respondents agreed that Cosion can speed up the diagnosis process.Additionally, Cosion has proven to offer a CAD system that can assist medical professionals in diagnosing cancer malignancies.It is also capable of accurately segmenting suspected cancer objects.
The features in Cosion performed well during testing.However, there were occasional errors due to unstable internet connections.The security aspect of Cosion was also considered weak in terms of accommodating patient data.The testing results for various aspects suggest that Cosion is worthy of implementation.However, further development is needed, such as conducting tests on datasets with noisy ultrasound images.Additionally, there is feedback to improve Cosion's compatibility with slower networks.
Unlike traditional CAD systems that require large machines, this web-based application can be accessed anywhere and anytime through a web browser on devices connected to the internet.Additionally, this web application allows for segmentation processes on a data repository, enabling remote diagnosis to be conducted.
Research on developing CAD systems with easy access, such as web-based systems, has not been extensively conducted.The study by (de Paula et al. 2010) developed a Java-based web application implemented for segmenting 50 Cardiac MRI images using the AC Snakes method in less than 5 seconds.In the research by (Pathan et al. 2022), an artificial intelligence-based method was used for rapid breast cancer detection in ultrasound images, achieving an accuracy of 81.02%.(Arslan Tuncer 2019) developed an Android-based program using a hybrid method (template matching and AC) for segmenting the Optic Disc, resulting in an accuracy of 0.90.Additionally, there is also a study by (Nugroho et al. 2023) that used the AC method called MoRbAC to develop a web application as part of a computer-aided design system.However, the web application focused on the detection stage, producing initial contours with an accuracy of 98.75%.
Apart from the differences in the objects being studied, several applications developed in these studies have not been directly tested on experts, so their implications in the medical field are not yet known.
The testing conducted mainly focused on the system's reliability in performing its tasks based on the methods used.
In this study, the segmentation method used is a method previously developed (HERBAC) that has been tested for accuracy and achieved a high value of 92.31%.Therefore, the testing in this research focuses more on the reliability of the developed web application, following the testing criteria of ISO 25010 software and incorporating validation aspects by the method expert (the creator of the HERBAC method), with a validation result of 77.5%.

Implications
The development of the Cosion web application has led to the creation of technological advancements in healthcare, particularly in CAD systems for cancer malignancy diagnosis.Cosion can assist healthcare professionals in performing segmentation processes more efficiently and accurately by avoiding human intervention, thus minimizing human errors.With the AC method called HERBAC adopted in Cosion, the segmentation process can run automatically.As a result, medical experts can perform their work more flexibly without being bound by time and location.By providing only a device connected to the internet, the segmentation process can be carried out.The user-friendly and easily accessible nature of the Cosion application is expected to help medical professionals speed up the process of cancer malignancy diagnosis, leading to faster patient management.

Conclusion
Based on the results of designing, building, and testing the Cosion web application for segmenting cancer objects on US images using the HERBAC method previously described, it can be concluded that the Cosion web application built in the context of developing a CAD system can be used to segment breast and thyroid cancer objects automatically.Cosion is deemed suitable for adopting all algorithms in the HERBAC method.This is supported by web testing conducted on five other aspects: functionality, portability, efficiency, reliability, and usability.The Cosion web application has demonstrated good performance in these aspects.Through usability testing, it has been determined that Cosion is capable of efficiently and user-friendly assisting radiologists in performing object segmentation of cancerous tissue in ultrasound images.
However, Cosion has several limitations.The main limitation is that it can only be used when connected to the internet.Therefore, in areas with limited internet access, Cosion cannot be utilized.Further development, such as creating an Android or desktop-based application, is required to address this issue.Additionally, Cosion has only been tested on datasets that confirmed thyroid and breast cancer cases.Further research is needed to evaluate the effectiveness of Cosion in segmenting cancerous objects in non-specific ultrasound images.Furthermore, Cosion is limited to the segmentation stage of the CAD system.It cannot provide a definitive conclusion on the malignancy of detected cancer objects.Cosion only assists radiologists in speeding up the diagnosis process by automatically performing object segmentation.
results (Source: https://youtu.be/Syu4NjD2qwA)CAD has been widely developed in several US machines, including the Ultrasound: S Detect for Thyroid RS80 developed by Samsung Medical Imaging as shown in Figure 1 point a (Samsung 2016).

Figure 8 .
Figure 8.The results of the HERBAC method program with (a) input, (b) ground truth, and (c) Cosion Jaccard coefficient of 0.989.The research by Babu et al. in 2021 utilized level set and Chan-Vese (CV) techniques for brain cancer segmentation using MRI (Babu et al. 2021).The results obtained showed a DC value of 97.80, a Hausdorff distance (HD) of 1.86, and a Jaccard similarity index (JSI) of 97.04.

Table 1 . Requirements Analysis and Specification
Portability testing is performed by simulating the use of the web on various devices connected to the internet through a browser.In this study, portability testing is limited to desktop devices running Windows and Mac OS, as well as mobile devices running Android and iOS with browsers including Internet Explorer, Microsoft Edge, Mozilla Firefox, Opera, Google Chrome, Safari, Android browser, and iPhone.3. The efficiency testing is conducted to measure the web performance using the basic parameters recommended by Google Developer for Page Speed and presented by Yahoo Developer Network through the GTmetrix website by entering the website address for each web page at https://GTmetrix.com/.In the efficiency testing, two values are obtained, which are the performance score and the structure score, and both scores are used to get the GTmetrix Grade.The GTmetrix Grade value is obtained from Equation 2 (GTmetrix 2020).