APL-Caching Approaches Comparison
This page contains complementary materials for the paper entitled An A Comparative Study of Application-level Caching Recommendations at the Method Level, authored by Rômulo Meloca and Ingrid Nunes.
The source code of our study is available for reproducibility at this git repository
Section 1 contains plots related to the performance improvement achieved by each approach, which were measured in the second phase of the experiment described in the paper.
In Section 2 we present plots of the statistics of each cached method.
In Section 3 we show the cache size along the execution.
Finally, given that APLCache recommends method-inputs, we also analysed the amount of inputs missed by it, which are presented in plots in Section 3.
In each section there are buttons to Show/Hide the respective plots.
1. Requests
In this section there are three plots.
On top, we present the median final throughput among all ten executions that we made for every application, measured in requests handled per second. Values were normalised and the absolute values appear on top of every bar. Error bars consist of the minimum and the maximum final throughput achieved.
Next, the left plot shows the cumulative throughput of each of the ten executions that we made for each approach & amount of simultaneous users. The most grouped the lines, the less variable are the application or the impact of adding the approach to it. In each page there are the results of each application.
The right plot compares the median execution (of the left plot) of each approach for each application.
2. Misses, Hits & Additions
In this section we present the amount of hits, misses and additions that each approach could achieve.
In the first plot we present these values separated per method. Each page presents results of each application. Each line of the plot present one recommended (and valid) method.
Its label is composed of the instance-type of recommendation (whether static for cache values shared among all instances of the class, or instance for cache values restricted per object instance of the class), followed by the input-type of the recommendation (getter if there are no inputs, single if the size of the cache shall be 1 or multi if the size of the cache is not restricted), and the name of the class and the method which was recommended.
There are intersections among approaches' recommendations.
For Memoizeit there are also numbers in the labels which represent the position that MemoizeIt ranked that recommendation. Numbers in the end of the label means that the recommended method was overloaded, and both were cached.
Note that not all misses are followed by an addition, specially for APLCache that first generate a miss before checking if the value is cacheable, in order to achieve better performance.
In the second plot we present the sum of hits, misses and additions that each recommendation achieved per approach. Values were normalised and the absolute values appear on top of every bar.
3. Cache size
In this section we present the cache size along the execution. Results are either presented aggregated and segregated by method.
4. Inputs
Finally, in this section we present the amount of inputs missed by APLCache. In the first page the results are split per method and in the second page they are summarized. Note that the numbers represent the number of occurrences, instead of the number of distinct inputs ignored. Distinct inputs are presented in the respective table in paper.