353 citations · 1.4k across the 27 of their papers we have counts for
29 papers
An efficient deep neural network to find small objects in large 3D images
Jungkyu Park, Jakub Chłędowski, Stanisław Jastrzębski +19
3D imaging enables accurate diagnosis by providing spatial information about organ anatomy. However, using 3D images to train AI models is computationally challenging because they…
Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks
Nan Wu, Stanisław Jastrzębski, Kyunghyun Cho +1
We hypothesize that due to the greedy nature of learning in multi-modal deep neural networks, these models tend to rely on just one modality while under-fitting the other modalitie…
Relative Molecule Self-Attention Transformer
Łukasz Maziarka, Dawid Majchrowski, Tomasz Danel +5
Self-supervised learning holds promise to revolutionize molecule property prediction - a central task to drug discovery and many more industries - by enabling data efficient learni…
Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization
Stanislaw Jastrzebski, Devansh Arpit, Oliver Astrand +6
The early phase of training a deep neural network has a dramatic effect on the local curvature of the loss function. For instance, using a small learning rate does not guarantee st…
Differences between human and machine perception in medical diagnosis
Taro Makino, Stanislaw Jastrzebski, Witold Oleszkiewicz +18
Deep neural networks (DNNs) show promise in image-based medical diagnosis, but cannot be fully trusted since their performance can be severely degraded by dataset shifts to which h…
RetroGNN: Approximating Retrosynthesis by Graph Neural Networks for De Novo Drug Design
Cheng-Hao Liu, Maksym Korablyov, Stanisław Jastrzębski +3
De novo molecule generation often results in chemically unfeasible molecules. A natural idea to mitigate this problem is to bias the search process towards more easily synthesizabl…
An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department
Farah E. Shamout, Yiqiu Shen, Nan Wu +17
During the coronavirus disease 2019 (COVID-19) pandemic, rapid and accurate triage of patients at the emergency department is critical to inform decision-making. We propose a data-…
We Should at Least Be Able to Design Molecules That Dock Well
Tobiasz Cieplinski, Tomasz Danel, Sabina Podlewska +1
Designing compounds with desired properties is a key element of the drug discovery process. However, measuring progress in the field has been challenging due to the lack of realist…
Molecule Edit Graph Attention Network: Modeling Chemical Reactions as Sequences of Graph Edits
Mikołaj Sacha, Mikołaj Błaż, Piotr Byrski +5
The central challenge in automated synthesis planning is to be able to generate and predict outcomes of a diverse set of chemical reactions. In particular, in many cases, the most…
Understanding the robustness of deep neural network classifiers for breast cancer screening
Witold Oleszkiewicz, Taro Makino, Stanisław Jastrzębski +5
Deep neural networks (DNNs) show promise in breast cancer screening, but their robustness to input perturbations must be better understood before they can be clinically implemented…
The Break-Even Point on Optimization Trajectories of Deep Neural Networks
Stanislaw Jastrzebski, Maciej Szymczak, Stanislav Fort +4
The early phase of training of deep neural networks is critical for their final performance. In this work, we study how the hyperparameters of stochastic gradient descent (SGD) use…
Molecule Attention Transformer
Łukasz Maziarka, Tomasz Danel, Sławomir Mucha +3
Designing a single neural network architecture that performs competitively across a range of molecule property prediction tasks remains largely an open challenge, and its solution…
Large Scale Structure of Neural Network Loss Landscapes
Stanislav Fort, Stanislaw Jastrzebski
There are many surprising and perhaps counter-intuitive properties of optimization of deep neural networks. We propose and experimentally verify a unified phenomenological model of…
Split Batch Normalization: Improving Semi-Supervised Learning under Domain Shift
Michał Zając, Konrad Zolna, Stanisław Jastrzębski
Recent work has shown that using unlabeled data in semi-supervised learning is not always beneficial and can even hurt generalization, especially when there is a class mismatch bet…
Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening
Nan Wu, Jason Phang, Jungkyu Park +29
We present a deep convolutional neural network for breast cancer screening exam classification, trained and evaluated on over 200,000 exams (over 1,000,000 images). Our network ach…
Non-linear ICA based on Cramer-Wold metric
Przemysław Spurek, Aleksandra Nowak, Jacek Tabor +2
Non-linear source separation is a challenging open problem with many applications. We extend a recently proposed Adversarial Non-linear ICA (ANICA) model, and introduce Cramer-Wold…
Parameter-Efficient Transfer Learning for NLP
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski +5
Fine-tuning large pre-trained models is an effective transfer mechanism in NLP. However, in the presence of many downstream tasks, fine-tuning is parameter inefficient: an entire n…
Stiffness: A New Perspective on Generalization in Neural Networks
Stanislav Fort, Paweł Krzysztof Nowak, Stanislaw Jastrzebski +1
In this paper we develop a new perspective on generalization of neural networks by proposing and investigating the concept of a neural network stiffness. We measure how stiff a net…
Neural Architecture Search Over a Graph Search Space
Stanisław Jastrzębski, Quentin de Laroussilhe, Mingxing Tan +3
Neural Architecture Search (NAS) enabled the discovery of state-of-the-art architectures in many domains. However, the success of NAS depends on the definition of the search space.…
Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function
Wojciech Tarnowski, Piotr Warchoł, Stanisław Jastrzębski +2
We demonstrate that in residual neural networks (ResNets) dynamical isometry is achievable irrespectively of the activation function used. We do that by deriving, with the help of…
On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length
Stanisław Jastrzębski, Zachary Kenton, Nicolas Ballas +3
Stochastic Gradient Descent (SGD) based training of neural networks with a large learning rate or a small batch-size typically ends in well-generalizing, flat regions of the weight…
Cramer-Wold AutoEncoder
Szymon Knop, Jacek Tabor, Przemysław Spurek +3
We propose a new generative model, Cramer-Wold Autoencoder (CWAE). Following WAE, we directly encourage normality of the latent space. Our paper uses also the recent idea from Slic…
Commonsense mining as knowledge base completion? A study on the impact of novelty
Stanisław Jastrzębski, Dzmitry Bahdanau, Seyedarian Hosseini +3
Commonsense knowledge bases such as ConceptNet represent knowledge in the form of relational triples. Inspired by the recent work by Li et al., we analyse if knowledge base complet…
Three Factors Influencing Minima in SGD
Stanisław Jastrzębski, Zachary Kenton, Devansh Arpit +4
We investigate the dynamical and convergent properties of stochastic gradient descent (SGD) applied to Deep Neural Networks (DNNs). Characterizing the relation between learning rat…