Publications by Tags

Capsule Networks, Convolutional Neural Networks, Counterfactual Detection, Dataset Curation, Exposure Bias, Information Retrieval, k-NN, Knowledge Distillation, Language Modeling, Meta-Embedding, Metric Learning, Multilingual Transformers, Neural Language Models, Neural Network Compression, Policy-Gradient Methods, Pruning, Quantization, Regularization, Reward Shaping, Supervised Learning, Tensor Decomposition, Text Generation, Textual Similarity Evaluation, Transformers, Weight Sharing

Capsule Networks

Siamese capsule networks

James O' Neill

arXiv preprint arXiv:1805.07242

TLDR: This paper proposes to extend capsule networks to a siamese network for metric learning tasks.

Capsule Networks, Metric Learning

Convolutional Neural Networks

Compressing deep neural networks via layer fusion

James O' Neill, Greg V. Steeg, Aram Galstyan

Asian Conference in Machine Learning

TLDR: This paper proposes a dynamic weight sharing technique that learns to tie weights during retraining (compression phase).

Convolutional Neural Networks, Neural Network Compression, Transformers, Weight Sharing

Counterfactual Detection

I Wish I Would Have Loved This One, But I Didn't–A Multilingual Dataset for Counterfactual Detection in Product Reviews

James O' Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota and Danushka Bollegala

Empirical Methods for Natural Language Processing (EMNLP)

TLDR: This paper proposes a dynamic weight sharing technique that learns to tie weights during retraining (compression phase).

Counterfactual Detection, Dataset Curation, Information Retrieval, Multilingual Transformers

Dataset Curation

I Wish I Would Have Loved This One, But I Didn't–A Multilingual Dataset for Counterfactual Detection in Product Reviews

James O' Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota and Danushka Bollegala

Empirical Methods for Natural Language Processing (EMNLP)

TLDR: This paper proposes a dynamic weight sharing technique that learns to tie weights during retraining (compression phase).

Counterfactual Detection, Dataset Curation, Information Retrieval, Multilingual Transformers

Exposure Bias

$ k $-Neighbor Based Curriculum Sampling for Sequence Prediction

James O' Neill, Danushka Bollegala

arXiv preprint arXiv:2101.09313

TLDR: We propose Nearest-Neighbor Replacement Sampling, a technique to mitigate exposure bias by replacing ground truth tokens with semantically similar tokens during training.

Exposure Bias, k-NN, Language Modeling

Transfer Reward Learning for Policy Gradient-Based Text Generation

James O' Neill, Danushka Bollegala

arXiv preprint arXiv:1909.03622

TLDR: We propose pretrained textual similarity models to issue rewards based on the semantic similarity of generated and ground truth sequences for an actor-critic sequence predictor.

Exposure Bias, Policy-Gradient Methods, Reward Shaping, Text Generation

Information Retrieval

I Wish I Would Have Loved This One, But I Didn't–A Multilingual Dataset for Counterfactual Detection in Product Reviews

James O' Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota and Danushka Bollegala

Empirical Methods for Natural Language Processing (EMNLP)

TLDR: This paper proposes a dynamic weight sharing technique that learns to tie weights during retraining (compression phase).

Counterfactual Detection, Dataset Curation, Information Retrieval, Multilingual Transformers

Knowledge Distillation

An Overview of Neural Network Compression

James O' Neill

arXiv preprint arXiv:2006.03669

TLDR: This paper provides a thorough overview of weight sharing, pruning, tensor decomposition, knowledge distillation and quantization.

Knowledge Distillation, Pruning, Quantization, Tensor Decomposition, Weight Sharing

Semantically-Conditioned Negative Samples for Efficient Contrastive Learning

James O' Neill, Danushka Bollegala

Asian Conference in Machine Learning

TLDR: This paper proposes a dynamic weight sharing technique that learns to tie weights during retraining (compression phase).

Knowledge Distillation, Neural Network Compression

Deep Neural Compression Via Concurrent Pruning and Self-Distillation

James O' Neill, Sourav Dutta, Haytham Assem

arXiv preprint arXiv:2109.15014

TLDR: This paper proposes the combination of pruning and self-distillation and uses a cross-correlation based KD objective that naturally fits with magnitude-based pruning.

Knowledge Distillation, Neural Network Compression, Pruning

Language Modeling

$ k $-Neighbor Based Curriculum Sampling for Sequence Prediction

James O' Neill, Danushka Bollegala

arXiv preprint arXiv:2101.09313

TLDR: We propose Nearest-Neighbor Replacement Sampling, a technique to mitigate exposure bias by replacing ground truth tokens with semantically similar tokens during training.

Exposure Bias, k-NN, Language Modeling

Meta-Embedding

Meta-embedding as auxiliary task regularization

James O' Neill, Danushka Bollegala

European Conference on Artificial Intelligence

TLDR: We propose supervised meta-embedding that learns to reconstruct an ensemble of static word embeddings while learning on a downstream task.

Meta-Embedding, Regularization, Supervised Learning

Metric Learning

Siamese capsule networks

James O' Neill

arXiv preprint arXiv:1805.07242

TLDR: This paper proposes to extend capsule networks to a siamese network for metric learning tasks.

Capsule Networks, Metric Learning

Multilingual Transformers

I Wish I Would Have Loved This One, But I Didn't–A Multilingual Dataset for Counterfactual Detection in Product Reviews

James O' Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota and Danushka Bollegala

Empirical Methods for Natural Language Processing (EMNLP)

TLDR: This paper proposes a dynamic weight sharing technique that learns to tie weights during retraining (compression phase).

Counterfactual Detection, Dataset Curation, Information Retrieval, Multilingual Transformers

Neural Language Models

Learning to Evaluate Neural Language Models

James O' Neill, Danushka Bollegala

Pacific Association of Computation Linguistics (PACLING)

TLDR: We propose pretrained textual similarity models to evaluate neural language models.

Neural Language Models, Textual Similarity Evaluation

Neural Network Compression

Semantically-Conditioned Negative Samples for Efficient Contrastive Learning

James O' Neill, Danushka Bollegala

Asian Conference in Machine Learning

TLDR: This paper proposes a dynamic weight sharing technique that learns to tie weights during retraining (compression phase).

Knowledge Distillation, Neural Network Compression

Deep Neural Compression Via Concurrent Pruning and Self-Distillation

James O' Neill, Sourav Dutta, Haytham Assem

arXiv preprint arXiv:2109.15014

TLDR: This paper proposes the combination of pruning and self-distillation and uses a cross-correlation based KD objective that naturally fits with magnitude-based pruning.

Knowledge Distillation, Neural Network Compression, Pruning

Compressing deep neural networks via layer fusion

James O' Neill, Greg V. Steeg, Aram Galstyan

Asian Conference in Machine Learning

TLDR: This paper proposes a dynamic weight sharing technique that learns to tie weights during retraining (compression phase).

Convolutional Neural Networks, Neural Network Compression, Transformers, Weight Sharing

Policy-Gradient Methods

Transfer Reward Learning for Policy Gradient-Based Text Generation

James O' Neill, Danushka Bollegala

arXiv preprint arXiv:1909.03622

TLDR: We propose pretrained textual similarity models to issue rewards based on the semantic similarity of generated and ground truth sequences for an actor-critic sequence predictor.

Exposure Bias, Policy-Gradient Methods, Reward Shaping, Text Generation

Pruning

An Overview of Neural Network Compression

James O' Neill

arXiv preprint arXiv:2006.03669

TLDR: This paper provides a thorough overview of weight sharing, pruning, tensor decomposition, knowledge distillation and quantization.

Knowledge Distillation, Pruning, Quantization, Tensor Decomposition, Weight Sharing

Deep Neural Compression Via Concurrent Pruning and Self-Distillation

James O' Neill, Sourav Dutta, Haytham Assem

arXiv preprint arXiv:2109.15014

TLDR: This paper proposes the combination of pruning and self-distillation and uses a cross-correlation based KD objective that naturally fits with magnitude-based pruning.

Knowledge Distillation, Neural Network Compression, Pruning

Quantization

An Overview of Neural Network Compression

James O' Neill

arXiv preprint arXiv:2006.03669

TLDR: This paper provides a thorough overview of weight sharing, pruning, tensor decomposition, knowledge distillation and quantization.

Knowledge Distillation, Pruning, Quantization, Tensor Decomposition, Weight Sharing

Regularization

Meta-embedding as auxiliary task regularization

James O' Neill, Danushka Bollegala

European Conference on Artificial Intelligence

TLDR: We propose supervised meta-embedding that learns to reconstruct an ensemble of static word embeddings while learning on a downstream task.

Meta-Embedding, Regularization, Supervised Learning

Reward Shaping

Transfer Reward Learning for Policy Gradient-Based Text Generation

James O' Neill, Danushka Bollegala

arXiv preprint arXiv:1909.03622

TLDR: We propose pretrained textual similarity models to issue rewards based on the semantic similarity of generated and ground truth sequences for an actor-critic sequence predictor.

Exposure Bias, Policy-Gradient Methods, Reward Shaping, Text Generation

Supervised Learning

Meta-embedding as auxiliary task regularization

James O' Neill, Danushka Bollegala

European Conference on Artificial Intelligence

TLDR: We propose supervised meta-embedding that learns to reconstruct an ensemble of static word embeddings while learning on a downstream task.

Meta-Embedding, Regularization, Supervised Learning

Tensor Decomposition

An Overview of Neural Network Compression

James O' Neill

arXiv preprint arXiv:2006.03669

TLDR: This paper provides a thorough overview of weight sharing, pruning, tensor decomposition, knowledge distillation and quantization.

Knowledge Distillation, Pruning, Quantization, Tensor Decomposition, Weight Sharing

Text Generation

Transfer Reward Learning for Policy Gradient-Based Text Generation

James O' Neill, Danushka Bollegala

arXiv preprint arXiv:1909.03622

TLDR: We propose pretrained textual similarity models to issue rewards based on the semantic similarity of generated and ground truth sequences for an actor-critic sequence predictor.

Exposure Bias, Policy-Gradient Methods, Reward Shaping, Text Generation

Textual Similarity Evaluation

Learning to Evaluate Neural Language Models

James O' Neill, Danushka Bollegala

Pacific Association of Computation Linguistics (PACLING)

TLDR: We propose pretrained textual similarity models to evaluate neural language models.

Neural Language Models, Textual Similarity Evaluation

Transformers

Compressing deep neural networks via layer fusion

James O' Neill, Greg V. Steeg, Aram Galstyan

Asian Conference in Machine Learning

TLDR: This paper proposes a dynamic weight sharing technique that learns to tie weights during retraining (compression phase).

Convolutional Neural Networks, Neural Network Compression, Transformers, Weight Sharing

An Overview of Neural Network Compression

James O' Neill

arXiv preprint arXiv:2006.03669

TLDR: This paper provides a thorough overview of weight sharing, pruning, tensor decomposition, knowledge distillation and quantization.

Knowledge Distillation, Pruning, Quantization, Tensor Decomposition, Weight Sharing

Compressing deep neural networks via layer fusion

James O' Neill, Greg V. Steeg, Aram Galstyan

Asian Conference in Machine Learning

TLDR: This paper proposes a dynamic weight sharing technique that learns to tie weights during retraining (compression phase).

Convolutional Neural Networks, Neural Network Compression, Transformers, Weight Sharing

k-NN

$ k $-Neighbor Based Curriculum Sampling for Sequence Prediction

James O' Neill, Danushka Bollegala

arXiv preprint arXiv:2101.09313

TLDR: We propose Nearest-Neighbor Replacement Sampling, a technique to mitigate exposure bias by replacing ground truth tokens with semantically similar tokens during training.

Exposure Bias, k-NN, Language Modeling