Holistic Combination of Structural and Textual Code Information for Context based API Recommendation

Chi Chen, Xin Peng, Zhenchang Xing, Jun Sun, Xin Wang, Yifan Zhao, and Wenyun Zhao X. Peng is the corresponding author. C. Chen, X. Peng, X. Wang, Y. Zhao and W. Zhao are with the School of Computer Science and the Shanghai Key Laboratory of Data Science, Fudan University, Shanghai, China, and Shanghai Institute of Intelligent Electronics & Systems, China. Z. Xing is with the Australian National University, Australia. J. Sun is with the Singapore Management University, Singapore.

Abstract

Context based API recommendation is an important way to help developers find the needed APIs effectively and efficiently. For effective API recommendation, we need not only a joint view of both structural and textual code information, but also a holistic view of correlated API usage in control and data flow graph as a whole. Unfortunately, existing API recommendation methods exploit structural or textual code information separately. In this work, we propose a novel API recommendation approach called APIRec-CST (API Recommendation by Combining Structural and Textual code information). APIRec-CST is a deep learning model that combines the API usage with the text information in the source code based on an API Context Graph Network and a Code Token Network that simultaneously learn structural and textual features for API recommendation. We apply APIRec-CST to train a model for JDK library based on 1,914 open-source Java projects and evaluate the accuracy and MRR (Mean Reciprocal Rank) of API recommendation with another 6 open-source projects. The results show that our approach achieves respectively a top-1, top-5, top-10 accuracy and MRR of 60.3%, 81.5%, 87.7% and 69.4%, and significantly outperforms an existing graph-based statistical approach and a tree-based deep learning approach for API recommendation. A further analysis shows that textual code information makes sense and improves the accuracy and MRR. We also conduct a user study in which two groups of students are asked to finish 6 programming tasks with or without our APIRec-CST plugin. The results show that APIRec-CST can help the students to finish the tasks faster and more accurately and the feedback on the usability is overwhelmingly positive.

Index Terms:

API, recommendation, deep learning, data flow, control flow, text

1 Introduction

In modern software development, developers heavily rely on APIs (Application Programming Interfaces). When developers do not know which API(s) to use for a desired feature, automatic API recommendation is an important way to help developers find the needed APIs effectively and efficiently. In general, API recommendation methods learn explicit or implicit API usage patterns from a large code base and then match partially written code with the patterns to recommend APIs. Existing methods differ in the types of code information they model and how they model code information.

Source code contains two core types of information: structural and textual. Structural code information, such as control and data flow, represents program logic which can be captured using a graph representation; textual code information, such as code comments, method names, variable names, reflects the semantics of the code in natural language. Take the code snippet in Fig. 1 as an example. Note that the correct API statement at line 8 should be $hashCode=str.hashCode()$ . The method name “computeHashCode” and the variable name “hashCode” reflect the intent of this method (assuming the proper tokenization of these names). The method body uses multiple APIs which implement three pieces of correlated program logics: 1) use a reader to read contents from a file line by line (line 3/4/5/6/11/12); 2) compute the hash code of the content (line 8); 3) add the hash value into a created list (line 2/7/9). These program logics can be modeled in a control and data flow graph as shown in Fig. 5. Note that variable names (e.g., “path”, “result”, “rd”, “br”, “str”, “hashCode”) are helpful for the understanding of relevant structural program logics.

For effective API recommendation, we need not only a joint view of both structural and textual code information, but also a holistic view of correlated API usage in control and data flow graph as a whole. Unfortunately, existing API recommendation methods exploit structural or textual code information separately. Based on the observation of linguistic naturalness of source code [1], many approaches [1, 2, 3, 4] have been proposed that rely on statistical language models for code auto-completion and API recommendation. The adopted statistical language models can be simple or enhanced n-gram model [1, 2, 3, 4] or complex deep learning models (e.g., Recurrent Neural Network (RNN)) [5, 6, 7]. No matter which types of statistical language models to use, these approaches treat code as a sequence of text tokens (which may sometimes be enriched with simple syntactic information such as program construct keywords and data types), but do not exploit structural code information of source code. As such, they cannot properly model the long-range dependencies between correlated but far-away API usage due to the limitation of the length of a sequence.

To overcome the limitation of token-sequence-based API recommendation, another important line of API recommendation methods [8, 9] analyze control and data flow graph for recommending APIs. However, these methods usually base their recommendation on the enumeration of control and data flow subgraphs, but lack a holistic view of the overall program logic. Consider the code snippet in Fig. 1. Fig. 3 shows nine control-and-data-flow subgraphs for this code snippet. Assume developers do not know the “java.lang.String.hashCode” API to be used at line 8. Unfortunately, existing methods recommend “java.io.BufferedReader.readLine” based on the fourth subgraph in Fig. 3 or “while” based on the sixth subgraph. Different subgraphs are treated independently for recommending relevant APIs. As smaller subgraphs usually appear more frequently than larger subgraphs, APIs from smaller subgraphs that capture only a partial aspect of the overall program logic often overshadow APIs from larger subgraphs that capture more holistic view of the program logic.

In this work, we propose a novel API recommendation approach called APIRec-CST (API Recommendation by Combining Structural and Textual code information), which addresses the limitation of independent modeling of structural and textual code information and the lack of holistic reasoning of code structure in existing API recommendation approaches. APIRec-CST is a deep learning model that combines the API usage with the text information in the source code based on an API Context Graph Network and a Code Token Network. As such, it can simultaneously learn structural and textual features for API recommendation. APIRec-CST uses an API context graph to model API usage in a control and data flow graph for the entire method, rather than independent partial subgraphs as in existing methods [8]. Our API context graph contains the holistic semantics of the API usage in the source code around the location for API recommendation. From this API context graph, the API Context Graph Network learns to extract informative structural features for API recommendation. The textual code information in the source code, such as method names, parameter names and variable names, is processed as a bag of code tokens which is fed into the Code Token Network to infer the developer’s intent jointly with the API Context Graph Network.

We conduct a series of experiments to evaluate the effectiveness of APIRec-CST. Our results show that APIRec-CST significantly outperforms an existing graph-based statistical approach and a tree-based deep learning approach for API recommendation. The overall top-1 accuracy of APIRec-CST is about 60.3%, the top-5 accuracy is about 81.5%, the top-10 accuracy is about 87.7% and the MRR is about 69.4%. In addition, our analysis shows that textual code information makes sense and improves the accuracy and MRR. The results of our user study with 18 students and 6 programming tasks show that APIRec-CST can help the students finish the tasks faster and more accurately and the feedback on our tool’s usability is overwhelmingly positive.

The main contributions of this work are as follows:

•

We propose an API recommendation approach called APIRec-CST that combines structural and textual code information in the source code by jointly learning a graph-based deep learning model and a token-based deep learning model for effective API recommendation.
•

We implement APIRec-CST as a tool that supports the efficient model training and API inference with GPU acceleration.
•

We evaluate the effectiveness of APIRec-CST for recommending APIs with both automatically constructed test instances and real programming tasks.

2 Motivation

We use the code examples in Fig. 1 and Fig. 2 to motivate the need for holistic combination of structural and textual code information for API recommendation. The example in Fig. 1 is to implement a method to compute the hash code of the content from a file line by line and then adds the computed hash code into a list. The developer has written the code he knows and needs help to complete the remaining code. The line marked as hole is the location that the developer requests the recommendation of proper APIs for computing the hash code of the content of a string.

Refer to caption — Figure 1: Example of Computing HashCode of Content from File

We can see that this program contains rich structural code information (i.e., multiple APIs and control and data flow among these APIs). We can get many subgraphs of different sizes according to control and data flow, such as the nine subgraphs shown in Fig. 3. Note that each subgraph is labeled with a serial number for the convenience of discussion. We do not list all the subgraphs for the code in Fig. 1 due to the space limitation. As we can see, each subgraph reflects partial program logic (semantics). For example, the seventh subgraph reflects the semantics of creating readers for reading a file. As another example, the fifth subgraph reflects the semantics of reading contents line by line. None of the subgraphs (including those not listed in the paper) independently can reflect the expected semantics (i.e., computing the hash code of a string) at the location of hole.

If the developer uses existing tools such as GraLan [8] that recommends APIs based on such subgraphs, he cannot get the correct API recommendation. Table I lists the top-10 recommendations by GraLan. The first column is the ranking of each recommendation. The second column lists the ten recommendations. The third column is the serial number of the subgraphs in Fig. 3 used as the parent graph based on which the corresponding recommendation is generated. In GraLan, each subgraph is considered as a context parent graph to generate child graphs (each child graph has one more node than its parent graph and the extra node is considered as a candidate API recommendation). From Table I, we can see that the top-10 recommendations by GraLan are generated based on partial program semantics and thus miss the correct recommendation.

In order to recommend the correct API, we need a holistic view of the overall program logic in the entire method. Hence, we represent the API usage in a whole control and data flow graph called API context graph (as shown in Fig. 5) instead of subgraphs for the entire method. The API context graph not only contains all semantics in subgraphs, but also integrates these semantics as a whole. The details of how to construct an API context graph will be introduced in Section 4.1. From the API context graph, we can see that it contains the following two major semantics: semantics-1) use a reader to read contents from a file line by line; semantics-2) add a value into a created list. Since these semantics are in one entire graph, they can be integrated to infer the semantics at the hole.

TABLE I: Top-10 Recommendations by GraLan [8] for the Code Snippet in Fig. 1

Rank	Recommendation	Parent Graph
1	java.util.List.add	1
2	java.util.ArrayList.new	2
3	java.io.BufferedReader.readLine	4
4	java.io.BufferedReader.new	5
5	while	6
6	java.io.BufferedReader.close	8
7	if	2
8	for	2
9	java.util.ArrayList.add	1
10	java.io.InputStreamReader.new	3

When observing these two semantics in a holistic view, we can find that the declared $String$ variable “str” is just used to store the content from the file but not used any more in semantics-1. Furthermore, the declared $int$ variable “hashCode” is not assigned a value in semantics-2. In addition, there lack of APIs to connect semantics-1 and semantics-2 to make the program logic complete. From this holistic view, we can infer that the semantics at the hole is to get a value of $int$ type based on some kind of processing of a variable of $String$ type. Note that the subgraph can be a whole graph in GraLan, but the larger a graph is, the less frequent it may occur in the training data which may cause the data sparsity issue. Our deep learning model learns a vector representation for each entire graph based on an information diffusion mechanism of all nodes and edges. In this way, each entire graph that has a distinct semantics will have a meaningful vector representation, no matter how large the graph is and how frequent the graph occurs in the code base. As such, our model does not suffer from the data sparsity issue.

However, we still cannot recommend the exact API needed at the $hole$ in Fig. 1, if we just consider the structural code information in this example. This is because we cannot decide what kind of processing should be performed on the variable of $String$ type. Let us see the code snippet in Fig. 2. The developer needs to implement a method to read scores stored in a file, convert each score to an integer and add it into a list for further use. We can see that the code in Fig. 2 is structurally very similar to the code in Fig. 1, because the program logics for reading file and list addition are the same. The API context graph of the code in Fig. 2 is the same as that of the code in Fig. 1, but the expected APIs at hole are different. If the developer requests API recommendation for these two code snippets, we should distinguish the different intents in the two code snippets. To that end, textual code information in code becomes very useful for inferring code intents. In Fig. 1, the method name “computeHashCode” and variable name “hashCode” imply that the processing on the variable of $String$ type is likely relevant to hash code processing. In Fig. 2, the method name “getIntegerScore” and variable name “score” can imply that the processing on the variable of $String$ type is likely relevant to String-Integer conversion.

To sum up, a joint view of both structural and textual code information and a holistic view of correlated API usage in control and data flow graph of the entire method is required for effective API recommendation.

3 Background

In this work, we adopt Graph Neural Networks (GNNs), in particular, Gated Graph Neural Networks (GG-NNs) [10], for API recommendation. An API usage can be naturally represented in the form of a graph where the nodes represent APIs and edges represent control/data flow between nodes. Furthermore, the nodes and edges can be labeled with additional context information, e.g., the nodes can be labeled with API calls and the edge labels can be used to distinguish control flow and data flow.

GNNs are a neural network model which take graph structures as inputs. GNNs are based on an information diffusion mechanism and work effectively for a variety of graphs, e.g., directed or undirected graphs and cyclic or acyclic graphs. In GNNs, each node of the graph corresponds to a unit. The unit captures the current state of a node and is used to compute the next state of the node when activated. The units update their states and exchange information until they reach a stable equilibrium [11]. The state of a node is composed of the label of the node, the labels of its incoming and outgoing edges and the states and labels of neighbor nodes with a parametric function. Formally, a state x_n(t) at the $t$ th iteration of a node n is defined as follows [11].

\textbf{x${}_{n}$}(t)=\textbf{f${}_{w}$}(\textbf{l${}_{n}$},\textbf{l${}_{co[n]}$},\textbf{x${}_{ne[n]}$}(t-1),\textbf{l${}_{ne[n]}$}),

(1)

where f_w is a parametric function, l_n is the label of node n, l_co[n] are the labels of edges containing node n, x_ne[n](t-1) are the states of nodes in the neighborhood of node n at the $(t$ $-$ $1)$ th iteration, and l_ne[n] are the labels of nodes in the neighborhood of node n. In this way, each node can get a node representation. Take the graph in Figure 4 as an example. The state x₁ of node 1 at time t is computed as x₁(t)=f_w(l₁,l_(1,2),l_(1,3),l_(1,4),x₂(t-1),x₃(t-1),x₄(t-1),l₂,l₃,l₄), where l₁ is the label of node 1, l_(1,2), l_(1,3), l_(1,4) are the labels of edges connected with node 1, x₂(t-1), x₃(t-1), x₄(t-1) are the states of the neighboring nodes (i.e., node 2, node 3 and node 4) of node 1 at time t-1 and l_(1,2), l_(1,3), l_(1,4) are the labels of these neighbors of node 1. The state of a node is connected with other nodes in the graph as nodes can communicate with each other based on the information diffusion mechanism. Through training, GNNs can be applied for subgraph matching, mutagenesis, and web page ranking [11].

GG-NNs [10] are based on GNN. The difference is that GNNs apply Almeida-Pineda algorithm [12, 13] for computing gradients, whereas GG-NNs apply back-propagation through time with Gated Recurrent Units [14] for computing gradients. GG-NNs use a soft attention mechanism to decide which nodes are more relevant to compute the final vector representation of the graph. The graph level representation vector x_g is computed as follows [10].

\textbf{x${}_{g}$}=tanh\Bigg{(}\sum_{n\in N}\sigma\Big{(}i(\textbf{x${}_{n}$}(t),\textbf{l${}_{n}$})\Big{)}\odot tanh\Big{(}j(\textbf{x${}_{n}$}(t),\textbf{l${}_{n}$})\Big{)}\Bigg{)},

(2)

where $\sigma$ (i(x_n(t),l_n)) works as a soft attention mechanism, $i$ and $j$ are neural networks taking as input the concatenation of x_n(t) and l_n and output real-valued vectors [10], and $\odot$ is element-wise multiplication.

To get a graph representation, GNNs require creating a dummy super node which is connected to all other nodes by a special type of edge [10]. Doing so in our context may destroy the structural code information of the source code itself. In addition, the soft attention mechanism of GG-NNs can help us to identify which nodes (i.e., APIs) in the API context graph are more important for API recommendation. In GG-NNs, the final representation of a graph is the accumulated information of each node with its importance computed through the soft attention mechanism. In this way, the final representation of a graph is a a holistic representation of all nodes. Therefore, we choose GG-NNs as our deep neural networks to learn the features of API context graphs from a holistic view. More details of GNNs and GG-NNs can be referred to [11, 10].

4 Approach

In this section, we present the detailed design of APIRec-CST. It takes a program with a hole as input, and outputs a ranked list of API recommendations for filling the hole.

4.1 Program Representation

Given a program with a hole, APIRec-CST first constructs an API context graph and a bag of code tokens. The API context graph is a graph representation of structural code information of the user-provided program, whereas the code tokens (including the method name, parameter names and variable names) capture the textual code information.

An API context graph is a directed graph $(N,E)$ where $N$ is a set of nodes and $E\subseteq N\times N$ is a set of edges. Each node in $N$ represents an API method call, an API field access, a variable declaration, an assignment, a control unit or a hole. Furthermore, each node is labeled differently according to its type. Table II shows how each type of node is labeled. We use a special node labeled with $Hole$ (called hole node hereafter) to represent the hole. There is an edge $(n,n^{\prime})\in E$ if and only if one of the following conditions is satisfied.

•

There is a direct control flow from $n$ to $n^{\prime}$ ;
•

There is a direct data flow from $n$ to $n^{\prime}$ ;
•

$n^{\prime}$ is the hole node and $n$ is a node representing the preceding statement in the program or $n$ is the hole node and $n^{\prime}$ is a node representing the subsequent statement in the program.

Given an edge $(n,n^{\prime})$ , we say that $n$ is the parent node of $n^{\prime}$ and $n^{\prime}$ is the child node of $n$ . In APIRec-CST, the edges in an API context graph are distinguished by labeling them with different types, i.e., an edge is labeled control flow (Type c) is there is direct control flow and no direct data flow; an edge is labeled data flow (Type d) is there is direct data flow and no direct control flow; an edge is labeled control and data flow (Type cd) is there are both direct control flow and direct data flow; and an edge is labeled special flow (Type s) if its source node or target node is the hole. Note that the special flow edge makes sure that the hole node is connected to its context.

Given a program, APIRec-CST systematically builds the API context graph statically. First, APIRec-CST builds the AST (i.e., Abstract Syntax Tree) of the program. Then it creates nodes and edges in the API context graph for each statement in the program based on the AST in the following way.

•

If the statement is an API method call, an API field access, a variable declaration or an assignment, a node is created according to the corresponding node type in Table II. Note that if the parameter of an API method call is also an API method call or an API field access, APIRec-CST first creates a node for the parameter.
•

If the current statement is an expression that includes several API method calls or API field accesses, APIRec-CST creates a node for each API method call or API field access one by one.
•

If the current statement is a control statement, APIRec-CST creates a node for the control unit according to its type and several other nodes together with edges connecting them as shown in Table 4.1. For example, if the current statement is a while statement, APIRec-CST first creates a While node, a Condition node, and a Body node. Two Type c edges are introduced, one from the While node to the Condition node and the other from the While node to the Body node.

TABLE II: Labels of Different Types of Nodes in API Context Graphs

Node Type	Label	Example
Vari. Decl.	[Full Class Name].Declaration	String str; $\rightarrow$ java.lang.String.Declaration
Vari. Decl. with Constant Assignment	[Full Class Name].Constant	String str = "str"; $\rightarrow$ java.lang.String.Constant
Vari. Decl. with Null Assignment	[Full Class Name].Null	String str = null; $\rightarrow$ java.lang.String.Null
Vari. Decl. with Object Creation	[Full Class Name].new([Parameter Types])	File file = new File(path); $\rightarrow$ java.io.File.new(java.lang.String)
API Method Call	[Full Method Name]([Parameter Types])	builder.append("str"); $\rightarrow$ java.lang.StringBuilder.append(java.lang.String)
API Field Access	[Full Field Name]	System.out; $\rightarrow$ java.lang.System.out
Cascading API Method Call (API Field Access)	[Full Method Name]([Parameter Types])	builder.append("str").toString(); $\rightarrow$
	.[Method Name]([Parameter Types])	java.lang.StringBuilder.append(java.lang.String).toString()
	[Full Field Name]	System.out.println("str"); $\rightarrow$
	.[Method Name]([Parameter Types])	java.lang.System.out.println(java.lang.String)
Nested API Method Call (API Field Access)	[Full Method Name]([Parameter Types])	writer.write(sb.toString()); $\rightarrow$ java.lang.StringBuilder.toString()
	[Full Method Name]([Parameter Types])	java.io.FileWriter.write(java.lang.String)
	[Full Field Name]	label.setForeground(Color.blue); $\rightarrow$ java.awt.Color.blue
	[Full Method Name]([Parameter Types])	javax.swing.JLabel.setForeground(java.awt.Color)
Control Unit	[Control Unit Name]	if $\rightarrow$ If

Control Statement Type	Node of Control Unit	Nodes and Edges
if statement	If	a Condition node and a Type c edge from If node to Condition node
		a Then node and a Type c edge from If node to Then node
		a ElseIf/Else node and a Type c edge from If node to ElseIf/Else node
while/do for/foreach statement	While/DoWhile For/Foreach	a Condition node and a Type c edge from While/DoWhile/For/Foreach node to Condition node
		a Body node and a Type c edge from While/DoWhile/For/Foreach node to Body node
switch statement	Switch	a Selector node and a Type c edge from Switch node to Selector node
		a series of Case nodes and Type c edges from Switch node to each Case node
		a Default node and a Type c edge from Switch node to Default node
try statement	Try	a series of Catch nodes and a Type c edge from Try node to the first Catch node
		Type c edges connecting Catch nodes in order (such as from first to second, from second to third)
		a Finally node and a Type c edge from the last Catch node to Finally node

Project	Model	Top-1	Top-5	Top-10	MRR
Galaxy	GraLan	29.4	60.5	73.4	42.2
(473)	Tree-LSTM	39.3	68.7	76.7	51.4
	APIRec-CST	51.0	81.6	88.2	63.6
JGit	GraLan	41.6	71.4	79.1	53.8
(4530)	Tree-LSTM	52.6	75.6	81.2	61.7
	APIRec-CST	66.4	85.1	89.5	74.2
Froyo-Email	GraLan	23.0	62.8	78.9	40.7
(1537)	Tree-LSTM	51.6	74.5	82.6	61.1
	APIRec-CST	63.7	86.0	91.3	73.5
Grid-Sphere	GraLan	36.5	66.5	80.9	48.6
(1847)	Tree-LSTM	48.0	72.4	80.6	58.3
	APIRec-CST	62.0	87.1	92.5	72.8
Itext	GraLan	19.6	64.3	75.7	37.4
(4444)	Tree-LSTM	46.0	68.1	75.6	55.4
	APIRec-CST	57.9	80.7	86.8	67.6
Log4j	GraLan	38.6	61.3	77.5	48.9
(2155)	Tree-LSTM	42.4	62.9	79.0	52.2
	APIRec-CST	50.6	67.7	79.2	58.4
Average	GraLan	31.5	64.5	77.6	45.3
	Tree-LSTM	46.7	70.4	79.3	56.7
	APIRec-CST	58.6	81.4	87.9	68.4

Project	Model	Top-1	Difference	Top-5	Difference	Top-10	Difference	MRR	Difference
Galaxy	APIRec-SO	46.9	+4.1	76.3	+5.3	82.2	+6.0	58.9	+4.7
(473)	APIRec-CST	51.0	+4.1	81.6	+5.3	88.2	+6.0	63.6	+4.7
JGit	APIRec-SO	61.7	+4.7	83.8	+1.3	88.6	+0.9	71.2	+3.0
(4530)	APIRec-CST	66.4	+4.7	85.1	+1.3	89.5	+0.9	74.2	+3.0
Froyo-Email	APIRec-SO	58.8	+4.9	82.4	+3.6	88.9	+2.4	68.7	+4.8
(1537)	APIRec-CST	63.7	+4.9	86.0	+3.6	91.3	+2.4	73.5	+4.8
Grid-Sphere	APIRec-SO	57.7	+4.3	83.1	+4.0	90.6	+1.9	69.0	+3.8
(1847)	APIRec-CST	62.0	+4.3	87.1	+4.0	92.5	+1.9	72.8	+3.8
Itext	APIRec-SO	56.3	+1.6	78.8	+1.9	84.4	+2.4	65.7	+1.9
(4444)	APIRec-CST	57.9	+1.6	80.7	+1.9	86.8	+2.4	67.6	+1.9
Log4j	APIRec-SO	48.5	+2.1	71.4	-3.7	83.7	-4.5	58.3	+0.1
(2155)	APIRec-CST	50.6	+2.1	67.7	-3.7	79.2	-4.5	58.4	+0.1
Overall	APIRec-SO	56.9	+3.4	80.0	+1.5	86.7	+1.0	66.8	+2.6
(14986)	APIRec-CST	60.3	+3.4	81.5	+1.5	87.7	+1.0	69.4	+2.6

Task	Group	avg	min	max	median	stan. dev.	p
T1	G1	888.0	485	1200	900	299.83	0.0904
T1	G2	680.0	188	1200	718	343.13	0.0904
T2	G1	671.4	232	1200	480	387.91	0.2677
T2	G2	562.4	246	1003	449	298.53	0.2677
T3	G1	1173.3	960	1200	1200	75.42	0.0001
T3	G2	441.3	160	703	463	150.04	0.0001
T4	G1	1159.6	836	1200	1200	114.39	0.0003
T4	G2	708.1	431	1154	697	211.87	0.0003
T5	G1	665.7	345	1200	558	295.51	0.0924
T5	G2	475.0	232	746	427	184.56	0.0924
T6	G1	1140.0	660	1200	1200	169.71	0.0013
T6	G2	707.3	255	1200	658	340.67	0.0013

Task	Group	avg	min	max	median	stan. dev.	p-value
T1	G1	0.26	0.00	1.00	0.00	0.41	0.0073
T1	G2	0.81	0.00	1.00	1.00	0.36	0.0073
T2	G1	0.47	0.00	1.00	0.33	0.41	0.1461
T2	G2	0.68	0.00	1.00	1.00	0.39	0.1461
T3	G1	0.11	0.00	1.00	0.00	0.31	0.0008
T3	G2	0.89	0.00	1.00	1.00	0.31	0.0008
T4	G1	0.11	0.00	1.00	0.00	0.31	0.0012
T4	G2	0.83	0.00	1.00	1.00	0.33	0.0012
T5	G1	0.36	0.00	1.00	0.38	0.37	0.0030
T5	G2	0.89	0.5	1.00	1.00	0.21	0.0030
T6	G1	0.04	0.00	0.33	0.00	0.10	0.0013
T6	G2	0.78	0.00	1.00	1.00	0.42	0.0013

Holistic Combination of Structural and Textual Code Information for Context based API Recommendation

Abstract

Index Terms:

1 Introduction

2 Motivation

3 Background

4 Approach

4.1 Program Representation

4.2 Architecture

4.3 Training Corpus Construction

5 Evaluation

5.1 Training Details

5.2 API Prediction Accuracy (RQ1)

5.3 Contribution of Textual Code Information (RQ2)

5.4 Effectiveness in Real Tasks (RQ3)

5.5 Qualitative Analysis

5.6 Threats to Validity

6 Related Work

7 Conclusion

References