[Meta-Graph] Meta-Graph: Few-Shot Link Prediction Using Meta-Learning

Time: Thursday 11-Jun-2020 16:00 (This is a past event.)

Discussion Facilitator:


Motivation / Abstract
Link prediction is a ubiqtuous task that can be applied to various real world scenarios including biomedical interaction networks, social networks and recommendation systems. The goal of link prediction is to learn from a graph to infer missing or previously unknown relationships. For instance, in a social network we may use link prediction to power a friendship recommendation system, or in the case of biological network data, link prediction might be used to infer possible relationships between drugs and diseases. However, previous work on link prediction generally focuses only on one particular problem setting: previous work generally assumes that link prediction is to be performed on a single dense graph, with at least 50% of the true edges observed during training. Bose and his co-authors investigate how to perform link prediction when only a sparse sample (less than 30%) of edges are available. The authors formulate link prediction as a few-shot learning problem and solve it via a multi-graph, meta-learning strategy. They experiment on 3 very different datasets and find that Meta-Graph has the strongest performance in the sparse data regime, acheiving new state of the art results on sparse graphs. 
Questions Discussed
- the difference between 'pre-training' and 'fine-tuning' in Meta-Graph
- the motivation for the meta-learning approach
- the rationale for VGAE as the baseline link prediction framework
- the importance of the graph signature function and what it represents 
- future directions in scaling Meta-graph to heterogeneous graphs/knowledge graphs
Key Takeaways
- Link prediction in sparse graphs using traditional methods both heuristic and embedding based suffer from poor performance
- Sparsity in GNNs is a subjective concept but for most domains, sparsity means we have access to less than 50% of the edges in our dataset 
- Meta-learning is a method of optimization that teaches our models the low level features that might be universal to graphs in a particular domain (example: all protein graphs might have the same set of low level features that can be used as an initialization point)
- In this manner, a global set of parameters is learnt that can be adapted/fine-tuned to specific tasks which are sparse graphs
- Optimization is done on a set of locally learnt parameters which in turn help to initialize the global parameter space 
- A graph signature function is computed using the global parameters which creates a layer-wise summary of the graph as a whole 
- This signature can be used to modulate the local parameters such that on inference, when a test graph is observed that looks similar to a train graph, the signature function is able to initialize the local parameters by one of three methods (gating, modulation and weights - see the paper for these details)

Stream Categories:
 Graph Neural Nets