For the best practices of computing VMAF at the right resolution, refer to our tech blog. video quality comparator base on vmaf and ffmpeg. Growth - month over month growth in stars. Manage code changesCalculation of video quality metrics, such as PSNR, APSNR, SSIM, DELTA, MSE, MSAD, VQM, NQI, VMAF and VMAF phone, VIF. 04. Source Code. easyVmaf - Python. With those two installed, you can build aom-av1-lavish with these steps (reference): aom-av1-lavish. Growth - month over month growth in stars. exe . Note: If you are looking for an equivalent solution but for live audio/video signals (IP streaming, HDMI, SDI, etc. Browse to AOMedia Git index and login with your account (Gmail credentials, for example). easyVmaf - Python. If you less than the recommended number of rungs, in either case, the quality loss. 更新pip; sudo -H pip install --upgrade pip. The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives. NOTE: The open source projects on this list are ordered by number of github stars. It predicts subjective video quality based on a. The progress bar should be at 50%. exe . It is a combination of several spatial and temporal metrics that are fused using machine learning models. Collect and Analyze Billions of Data Points in Real Time. and uses scripting based frame servers, AviSynth+ and VapourSynth, for video processing. ts (blue arrow on the. , the first frame of distorted-A. VMAF is a valuable metric for estimating video quality, but it has its limitations. example: --vmaf_path "vmaf_v0. With version 14, navigating in the Result plot was instantaneous, no different than working with a 10-second test file. A compressor is mainly used to reduce the dynamic range of a signal. It supplies only GUI edition for Windows and has limitations. all the videos encoded to 10000 kbps. A fork of aom-av1-psy, which is a fork of aomenc. Setup the perfect encode for your video and add it to a queue. The VMAF models are actually trained by humans who rated the quality on a 1-5 point scale, watching the movie either on an 1080p screen from 3x screen height away, or on a 4k screen from 1. The latest version of “vmaf” needs an x86 assembler in order to build, so one. Av1an - Cross-platform command-line AV1 / VP9 / HEVC / H264 encoding framework with per scene quality encoding. The yellow line represents low-frame VMAF scores, or the lowest frame in the encoded file, a harbinger of transient quality issues. The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives. Adaptive extra splits by @master-of-zen in #584. The aim was to find the best possible settings for me between encoding time and quality. These instructions focus on using the inspector with AOMAnalyzer, but all tools can be built with Emscripten. . But I seem to have lower standards than a lot of people on this sub. mkv -filter_complex libvmaf -f null - but I'm getting this error: img2. 0 C++ vmaf VS video-compare Split screen video comparison tool using FFmpeg and SDL2In the case of the default VMAF model (model/vmaf_float_v0. More than 100 million people use GitHub to discover, fork, and contribute to over 330 million projects. I couldn't believe my eyes when I saw it. About. Download the . there is no many setting, so i don't know if i did a mistake, or that the reality is weird, or that the VMAF GUI have some bugs. VMAF-Master was originally written for Linux and the Mac with the Windows executable supplied by some open-source contributors. is a command line tool which supports VMAF feature extraction and prediction. Personally, I consider a VMAF score over 95 to be “good enough” for my use cases. Now that we know why we need metrics like VMAF and PSNR, we can take a closer look on how to calculate them. Growth - month over month growth in stars. x supports JSON-based model files only. If you have a CPU with some of the more recent instruction sets, like AVX-512, then you can get massive speed increases using Intel's SVT encoders. 0 2,163 10. Distorted file have index=0 Reference file have index=1. mpg -lavfi libvmaf -f null -. then get a command prompt into the directory you're running vmaf-gui and run the following command . Trial license is an edition for companies considering purchasing VQMT. 1. 0. Links:FFmpeg: Master: The vmafossexec command line tool has been deprecated and replaced with the more flexible and powerful vmaf tool. json. FFmpeg-Builds - FFmpeg Builds for yt-dlp . 1. vmaf video-quality video-quality-assessment vmaf-gui ffmpeg-vmaf Updated Aug 8, 2023; pavancm / LIVE-HFR Star 3. It supports commonly used quality metrics, such as PSNR, SSIM, VMAF, and modern standards (AVC/H. ab-av1 is using an estimated mean-VMAF of the whole file, rather than a per-chunk VMAF. This page attempts to maintain a (non-exhaustive) list of references on VMAF, including tech blogs, academic papers, presentations, etc. I tried to make some VMAF testing with the VMAF GUI. It works if I run it on the system it was built on. conf (see FFMetrics. The api returns a similarity percentage for each face with one another and the bounding boxes for each face. Player - ️ Play and. This is the best answer by far. It varies by codec implementation. Activity is a relative number indicating how actively a project is being developed. A tool to make encoding faster and easier for AV1 encoders. The yellow rows are the rungs of the ladder selected as described below. Feb 12. It is a FFMpeg GUI, so all calculations are done by FFMpeg. Build Using visual studio Just click the . The starting material is a film clip about 20 minutes long, 3 GB in size and with an average bandwidth of about 18 MBit, encoded in h264. Once you get the . y4m or . Recent commits have higher weight than older. VMAF is a codec-independent objective metric of comparable quality between source and output, using USC & Netflix's (and others) algorithm, which attempts to produce a model-based metric that is superior to other. ts. vmaf-gui - A GUI built using C# to make Netflix's VMAF easier to use digital_video_introduction - A hands-on introduction to video technology: image, video, codec (av1, vp9, h265) and more (ffmpeg encoding). zip并解压，windows在cmd窗口下进入到G:测试源分享vmaf-masterx64Release目录，然后运行vmafossecex. GUI for AV1 (aomenc, rav1e & svt-av1). mkvtools - bash tools for working with mkv files. Definitely the most precise tool to benchmark video encoders. vmaf-gui has no bugs, it has no vulnerabilities,. does that make any sense to you? the x265 nearly. Activity is a relative number indicating how actively a project is being developed. v1. SonarLint. More than 100 million people use GitHub to discover, fork, and contribute to over 330 million projects. A VMAF score is simpler to understand because it operates in a linear range of 0 to 100, whereas PSNR is logarithmic. It then renders the chart to a buffer and saves the resulting image as 'vmaf-graph. NET . This program has built-in support for the following models: vmaf_v0. Calculate various video quality metrics with FFmpeg. Install FFmpeg. Since libvmaf v2. A GUI built using C# to make Netflix's VMAF easier to use. Compare. libvmaf version 2. exe --threads 4 --reference. VMAF was designed for 1080p at an expected seating distance relative to display height of 3H. 4 extracted from 7z FileSince posting about ab-av1 I’ve been improving the tool. Repeat for as many videos as you want to encode. 1. Stars - the number of stars that a project has on GitHub. Learn more. {"payload":{"allShortcutsEnabled":false,"fileTree":{"vmaf-gui":{"items":[{"name":"Properties","path":"vmaf-gui/Properties","contentType":"directory"},{"name. NET FFMpeg/FFProbe wrapper for easily integrating media analysis. Growth - month over month growth in stars. VMAF computation for two video samples, reference. FFMetrics is a FFMpeg GUI that purpose is to visualize quality metrics calculated by FFMpeg. g. pkl. It's important not to forget this because these files are deleted after the log file is generated. 6. vmaf-gui - A GUI built using C# to make Netflix's VMAF easier to use . LibHunt /DEVs. FFmpeg - . fix indicatif by @master-of-zen in #583. NET Library for WinUI 3/ WPF/WinForms (based on FFmpeg/DirectX) Audio Switcher - . FFMpeg can be used for calculating different visual quality metrics (PSNR, SSIM, VMAF). When you are done, save the queue to a file for later, or run it now with the encode button. NotEnoughAV1Encodes - GUI for AV1 (aomenc, rav1e & svt-av1) . NET Library which facilitates interacting with Audio Devices on Windows SeeShark - Simple C# camera library. 'vmaf-gui - A GUI built using C# to make Netflix's VMAF easier to use . 264 MKV files. Command: auto-encode. windows encoding gui h264 video ffmpeg mpv comparison hevc x264 x265 vp9 av1 aom svt-av1 Updated Nov 14, 2023; C#; divyam234 / ffmpeg-NVEnc Star 37. 8 and 74. A GUI built using C# to make Netflix's VMAF easier to use. net. 2. NET Library for WinUI 3/ WPF/WinForms (based on FFmpeg/DirectX) NotEnoughAV1Encodes - GUI for AV1 (aomenc, rav1e & svt-av1) FF-Video-Converter - Simple video converter with cutting and cropping functionality, powered by the ffmpeg. 安装依赖库; pip install --user numpy scipy matplotlib pandas scikit-learn scikit-image h5py sureallibaom, the AOMedia codec in FFmpeg, v 2. ts, i. This software package includes a stand-alone C library libvmaf and its wrapping Python library. FFMetrics - Visualizes Video Quality Metrics (PSNR, SSIM & VMAF) calculated by ffmpeg. 0 forks Report repository Releases. conf. The configure output will show the audio filters included in your build. Discussion. v1. This can be mitigated by using more samples. Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars. All necessary functionality for effective in-depth analysis of encoded video sequences. The yellow line represents low-frame VMAF scores, or the lowest frame in the encoded file, a harbinger of transient quality issues. 8725, VMAF=67. The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives. 1. If any videos are in the queue, qencoder will encode them. . ts is delayed in comparition with distorted-A. Other Options. 0 API as well as an explanation of the new vmaf tool, please see the following README files: libvmaf, vmaf. To calculate VMAF value for current frame it is needed to use the previous frame and the next frame. FastFlix - FastFlix is a free GUI for HEVC and AV1 encoding, GIF/WebP/AVIF creation, and more!. Welcome to vsdb. Recent commits have higher weight than older. The green box is the cell identified via conditional formatting as equalling the Max score which is the highest VMAF score at that resolution. The tool takes a pair of input videos as well as a trained VMAF model and writes an output log containing per-frame and pooled VMAF scores. Be aware, that VMAF's libvmaf library will soon be dropped and replaced with a new library called libvmaf_rc. 19% of the quality of the Very Slow preset in 0. pkl – path to VMAF model (this is the default model):log_path=VMAF. exe . 0 C# vmaf-gui VS mpv. VMAF is a machine learning based video quality assessment method, originally designed for streaming applications, which combines multiple quality metrics and video features through SVM regression. staxrip - 🎞 Video encoding GUI for Windows. all the tests are very same. ffmpeg. Activity is a relative number indicating how actively a project is being developed. 727450. --vmaf-path <VMAF_PATH> Path to VMAF model (used by --vmaf and --target-quality) If not specified, ffmpeg's default is used. pkl" Recomended to place both files in encoding folder (`vmaf_v0. I couldn't believe my eyes when I saw it. How to VMAF (with ffmpeg), journey to the center of despair-- VMAF installation guide on Windows ; VMAF in FFmpeg – Installation and Usage Guide for Ubuntu-- VMAF installation guide on Ubuntu GUI Visualization Tools ; VideoBench-- VMAF, PSNR and bitrate analyzer ; FFMetrics-- Windows-centric GUI for PSNR, SSIM and VMAF. In this article, guest author Gabriel Dávila Revelo introduces us to a tool ( easyVMAF) that he’s developed to make VMAF computations easier and straightforward. Run at any scale in any environment in the cloud, on-premises, or at the edge. 2 41 10. Step 2: Install the Video and Audio Codecs that you are going to need. FFMetrics - Visualizes Video Quality Metrics (PSNR, SSIM & VMAF) calculated by ffmpeg. I saw AV1 encode with NVENC listed on the specs when it was released and. 6. It was primarily developed because the wide and diverse amount of content available from Netflix did not work well with any existing image quality assessment algorithm. vmaf video-quality video-quality-assessment vmaf-gui ffmpeg-vmaf. Download latest release from Github. A GUI built using C# to make Netflix's VMAF easier to use. - GitHub - Clybius/aom-av1-lavish: A fork of aom-av1-psy, which is a fork of aomenc. VMAF is a full reference metric that compares the reference (or source) and distorted video sequences to predict the subjective video quality. FastFlix is a handy GUI for encoding videos. mp4 -filter_complex libvmaf -f null -. Compare vmaf-gui vs video-quality-metrics and see what are their differences. Recent commits have higher weight than older. VMAF is a tool developed by Netflix to grade the quality of a compressed video file in comparison to the Original. 1. Activity is a relative number indicating how actively a project is being developed. Which is the best alternative to vmaf-gui? Based on common mentions it is: Video-quality-metrics, FFMpegCore, easyVmaf, NotEnoughAV1Encodes or Xabe. The uncompressed file is already at 306GB!. 0 C# AV1/HEVC/VP9/H264 parallel encoder GUI for FFmpeg with preview/comparisonUse the --vmaf-model-path option to set the path to a different VMAF model file. exe . FastFlix - FastFlix is a free GUI for HEVC and AV1 encoding, GIF/WebP/AVIF creation, and more!. A GUI built using C# to make Netflix's VMAF easier to use (by ThatNerdUKnow) #Video #GUI #Vmaf #Ffmpeg #Ssim #Psnr #vmaf-gui. Activity is a relative number indicating how actively a project is being developed. 265, VVC/H. Works with almost all video codecs (mpeg2, H264, HEVC, VP9. 0. pkl. It seemed to build correctly and the c:vmaf-install directory now contained three subdirectories, bin, include and lib. exe on another system fails due to the missing. 1. vmaf-gui - A GUI built using C# to make Netflix's VMAF easier to use . fifonik. VMAF is a machine learning based video quality assessment method, originally designed for streaming applications, which combines multiple quality metrics and video features through SVM regression. It is important how video streams supplied to libvmaf filter. In terms of really exploring the differences between the files, VQMT has a much richer and more usable feature set, as you would expect from a 14th-generation tool. 0. Growth - month over month growth in stars. The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives. And in the same note, I've added references to hardware. FastFlix - FastFlix is a free GUI for HEVC and AV1 encoding, GIF/WebP/AVIF creation, and more! 使用vmaf笔记 安装vmaf. GitHub is where people build software. 0 API as well as. exe . The VMAF models are actually trained by humans who rated the quality on a 1-5 point scale, watching the movie either on an 1080p screen from 3x screen height away, or on a 4k screen from 1. To. 7z. nmkoder - Media encoding, muxing, analysis toolkit for Windows . VMAF: vmaf: VMAF (Video Multi-Method Assessment Fusion) is a perceptual video quality assessment algorithm developed by Netflix. I evaluated this enhancement after measuring VMAF on a 105-minute movie. 6. More than 100 million people use GitHub to discover, fork, and contribute to over 330 million projects. 1-full_build-ffmpeg-release-full. Contrary to what I've read about rav1e, in my experience I have had fantastic quality (VMAF in the high 90s and 100) and up to 80% savings on bitrate in my tests of some personal videos (family videos from Android phones, so H. video-quality-metrics - Test specified presets/CRF values for the x264 or x265 encoder. vmaf (requires vim) (make) xtrabackup (requires vim) (make) diffoscope (requires vim) (check. --vmaf_plots Make plots for target quality search decisions (Exception: early skips) Saved in temp folder --vmaf_path Custom path to libvmaf models. Alkl58/VMAF-Plot-GUI. staxrip - 🎞 Video encoding GUI for Windows. Activity is a relative number indicating how actively a project is being developed. VMAF is a perceptual video quality assessment algorithm developed by Netflix. Manage all types of time series data in a single, purpose-built database. ffmpeg -i main. 1. Manage code changesmeson setup libvmaf libvmaf/build --buildtype release --default-library static --prefix C:/vmaf-install. 1 watching Forks. I tested three files totaling about 8. Video Multi-Method Assessment Fusion (VMAF) is an open source full-reference video quality assessment algorithm developed by Netflix in 2016. vmaf-gui - A GUI built using C# to make Netflix's VMAF easier to use Flyleaf - Media Player . In the build folder you’ll have to provide your own copy of ffmpeg, vmaf, and the model files for vmaf. It is a FFMpeg GUI, so all calculations are done by FFMpeg. 2 Answers Sorted by: 6 It does not really mater how you are specifying source files in command line. Manage code changesThis list will help you: dssim, FUnIE-GAN, iSeeBetter, piqa, ssim, video-quality-metrics, and vmaf-gui. mp4 -i original. json. 5. Install vpx VP9, VP8 encoding. Two model files are provided, vmaf_v0. Recent commits have higher weight than older. It is equal to [0] [1]libvmaf. There is also the phone model that can be enabled by using the -pm argument. VMAF- Invented by Netflix and then open-sourced, VMAF is widely available. json and vmaf_4k_v0. staxrip - 🎞 Video encoding GUI for Windows. Host and manage packages. 2. ). ThatNerdUKnow. staxrip - 🎞 Video encoding GUI for Windows. For VMAF and butteraugli, you'll need libvmaf and libjxl built and installed, as described in the README. If you're like me, you find the functionality of VMAF very useful but very time consuming and confusing to use at first. But in general I find my test videos consistent enough that sampling with a just a few ends up fairly accurate. VMAF has reliable support, uses a high-quality codebase, and has functional tools and libraries for enhanced results. Compares VMAF/SSIM/PSNR numerically & via graphs. Hence, a higher number means a better vmaf-gui alternative or higher similarity. Growth - month over month growth in stars. GitHub is where people build software. gui video ffmpeg ssim psnr vmaf vmaf-gui Updated Jul 27, 2022; C#; x-stars / ImageQuality Star 8. Contrary to what I've read about rav1e, in my experience I have had fantastic quality (VMAF in the high 90s and 100) and up to 80% savings on bitrate in my tests of some personal videos (family videos from Android phones, so H. metrics for each plane (Y, U, V) or components/submetrics (in the case of VIF, VMAF) global statistics (min/max/average/standard deviation) Author: Werner Robitza werner. Ideally, choose a portion of the video with more action and changing scenes. FFmpeg官网提供编译好的Windows程序不包含VMAF的评价工具，想要使用libvmaf需要同时下载ffmpeg和libvmaf的源代码然后自行编译。命令行类似于前两个。 或许Windows端也可以使用Docker（？）不知道Docker可否解决编译的难题？不管了，我用不着，各位有缘人自己看着办吧=_= A GUI built using C# to make Netflix's VMAF easier to use Av1ador. nmkoder - Media encoding, muxing, analysis toolkit for Windows . Activity is a relative number indicating how actively a project is being developed. 1. 100 by default. Av1an - Cross-platform command-line AV1 / VP9 / HEVC / H264 encoding framework with per scene quality encoding . video-quality-metrics - Test specified presets/CRF values for the x264 or x265 encoder. Is it just that these encoders can't compete with the. 6. Compare. x supports JSON-based model files only. In the Subjectify ratings, the lowest quality clip rated 64% the quality of the highest, closest to VMAF’s scoring. GitHub is where people build software. model file as well or else vmaf will fail). vmaf-gui - A GUI built using C# to make Netflix's VMAF easier to use . Compares VMAF/SSIM/PSNR numerically & via graphs. More than 94 million people use GitHub to discover, fork, and contribute to over 330 million projects. During the testing process, I sent questions and encoding strings to Google, which was kind enough to lend assistance. NET FFMpeg/FFProbe wrapper for easily integrating media analysis and conversion into your C# applications . The VMAF framework allows training of prediction models tailored to specic viewing conditions, no matter whether it is on a mobile phone or on a UHD TV. Each project can also be extended by various events and settings. It will probably be slower in the initial rendering. 264 with mediocre quality at 1080p). gui video ffmpeg ssim psnr vmaf vmaf-gui Updated Feb 16, 2023; C#; amanchadha / FRVSR-GAN Star 50. Next, follow the Generate Password Password link at the top of the page. A) YT Bitrate. It allows to deinterlace, scale and sync Ref and Distorted video automatically. However, VMAF is only available through the commandline or as an extension to FFMPEG. video ffmpeg vmaf video-quality Updated May 11, 2023;Never target vmaf 100, vmaf is a terrible metric for near-transparency. Code Issues Pull requests Evaluate image quality by objective index. ts matchs with the frame located at 0. json Use the 4k version if you have a 4K reference sample. Stars - the number of stars that a project has on GitHub. nmkoder - Media encoding, muxing, analysis toolkit for Windows . (fast) 2pass-quarter. Hence, a higher number means a better Xabe. Compares VMAF/SSIM/PSNR numerically & via graphs. Recent commits have higher weight than older. 77a93fd. 4. EDIT: I am using the latest version 1. 安装依赖库; pip install --user numpy scipy matplotlib pandas scikit-learn scikit-image h5py sureal libaom, the AOMedia codec in FFmpeg, v 2. The advantage of. exe . vmaf-gui is a C# library typically used in Telecommunications, Media, Media, Entertainment, Video, Video Utils, Pytorch applications. 6. 727450. But I seem to have lower standards than a lot of people on this sub. json to that folder, it should look like this:. 6. However, just like with transcoding, this process takes a lot of time to run sequentially on your own machine. The next step is to create a VmafModel. ToolsThe number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives. all the videos encoded to 10000 kbps. "model_path=vmaf_v0. You can download F. Hopefully, its compilation with FFmpeg will remain smooth as well. VMAF is an on-going project. Overall, FFMetrics is a very simple way to compute VMAF, PSNR, and SSIM, and graphically display the results of up to 12 files. Video quality assessment (VQA) is now a fast-growing subject, maturing in the full reference (FR) case, yet challenging in the exploding no reference (NR) case. -. FFMetrics - Visualizes Video Quality Metrics (PSNR, SSIM & VMAF) calculated by ffmpeg. ab-av1 auto-encode [OPTIONS] -i <INPUT> --preset <PRESET>. FastFlix - FastFlix is a free GUI for HEVC and AV1 encoding, GIF/WebP/AVIF creation, and more!. Video Bench is a tool that makes it easy to compare transcoded video files with a reference file (source) Calculates VMAF and PSNR scores for test files. vmaf-gui - A GUI built using C# to make Netflix's VMAF easier to use FastFlix - FastFlix is a free GUI for HEVC and AV1 encoding, GIF/WebP/AVIF creation, and more! nmkoder - Media encoding, muxing, analysis toolkit for Windows easyVmaf - Python script to easily compute VMAF using FFmpeg. It can be interpreted that the same distorted video would be perceived as having a higher quality when viewed on a phone screen than on a HD TV, and that the VMAF score differences between the 720p and 1080p videos are smaller using the phone model. IE: Your eyes are 3x the distance from the display as it's height. This is much faster than trying to process an entire movie or tv show through VMAF/PSNR. Video Multimethod Assessment Fusion ( VMAF) is an objective full-reference video quality metric developed by Netflix in cooperation with the University of Southern California, The IPI/LS2N lab Nantes Université, and the Laboratory for Image and Video Engineering (LIVE) at The University of Texas at Austin. Write better code with AI Code review. This is where VMAF shows a tremendous advantage, as illustrated below. 0 C# FFMpegCore VS vmaf-gui A GUI built using C# to make Netflix's VMAF easier to use Flyleaf-548 0. Vmaf is only available as a command line application and you have to learn how to use ffmpeg to. Edit details. Learn more. 1. Later in v0. The Max column identifies the highest score for each data rate. AV1AN: Updated build and integrated the new chunk order option into GUI; AV1AN: If the output directory does not exist, Nmkoder will try to create it before starting av1an; Track. Set required for input by @redzic in #585. When comparing NotEnoughAV1Encodes and vmaf-gui you can also consider the following projects: Av1an - Cross-platform command-line AV1 / VP9 / HEVC / H264 encoding framework with per scene quality encoding. 1. Hopefully, its compilation with FFmpeg will remain smooth as well. 6. And, remember to create the following folders. 0 C# NotEnoughAV1Encodes VS Av1ador AV1/HEVC/VP9/H264 parallel encoder GUI for FFmpeg with. AV1AN: Target VMAF mode may not work with SVT-AV1; Per-track audio settings do not work correctly when muxing from multiple input files (still) Installation: Download; Unzip anywhere using 7zip (where you don't need administrator permissions - Don't use Program Files) Run Nmkoder. Compare. exe . In this article, we compare and analyze the performance of the SVT-AV1 presets by varying the CRF values and showing how the objective quality (PSNR, VMAF, SSIM), file size, and encoding speed vary. Get Metrics: Calculate quality metrics like VMAF, SSIM, PSNR; Transfer Color Metadata: Copy. , downscaled, noisy and compressed images). Repeat for as many videos as you want to encode. Ideally, choose a portion of the video with more action and changing scenes. vmaf video-quality video-quality-assessment vmaf-gui ffmpeg-vmaf Updated Aug 8, 2023; katiashh / UAP_Attack_on_Quality_Metrics Star 11. exe .