Empirical Inference


2024


no image
Demonstration: Minsight - A Soft Vision-Based Tactile Sensor for Robotic Fingertips

Andrussow, I., Sun, H., Martius, G., Kuchenbecker, K. J.

Hands-on demonstration presented at the Conference on Robot Learning (CoRL), Munich, Germany, November 2024 (misc) Accepted

Abstract
Beyond vision and hearing, tactile sensing enhances a robot's ability to dexterously manipulate unfamiliar objects and safely interact with humans. Giving touch sensitivity to robots requires compact, robust, affordable, and efficient hardware designs, especially for high-resolution tactile sensing. We present a soft vision-based tactile sensor engineered to meet these requirements. Comparable in size to a human fingertip, Minsight uses machine learning to output high-resolution directional contact force distributions at 60 Hz. Minsight's tactile force maps enable precise sensing of fingertip contacts, which we use in this hands-on demonstration to allow a 3-DoF robot arm to physically track contact with a user's finger. While observing the colorful image captured by Minsight's internal camera, attendees can experience how its ability to detect delicate touches in all directions facilitates real-time robot interaction.

Project Page [BibTex]

2024

Project Page [BibTex]


no image
Learning to Control Emulated Muscles in Real Robots: Towards Exploiting Bio-Inspired Actuator Morphology

Schumacher, P., Krause, L., Schneider, J., Büchler, D., Martius, G., Haeufle, D.

In 10th International Conference on Biomedical Robotics and Biomechatronics (BioRob), September 2024 (inproceedings) Accepted

arXiv [BibTex]

arXiv [BibTex]


no image
Moûsai: Efficient Text-to-Music Diffusion Models

Schneider, F., Kamal, O., Jin, Z., Schölkopf, B.

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), Volume 1: Long Papers, pages: 8050-8068, (Editors: Lun-Wei Ku and Andre Martins and Vivek Srikumar), Association for Computational Linguistics, August 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Modelling Variability in Human Annotator Simulation

Wu*, W., Chen*, W., Zhang, C., Woodland, P. C.

Findings of the Association for Computational Linguistics (ACL), pages: 1139-1157, (Editors: Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek), Association for Computational Linguistics, August 2024, *equal contribution (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals

Ortu*, F., Jin*, Z., Doimo, D., Sachan, M., Cazzaniga, A., Schölkopf, B.

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , Volume 1, Long Papers, pages: 8420-8436, (Editors: Lun-Wei Ku and Andre Martins and Vivek Srikumar), Association for Computational Linguistics, August 2024, *equal contribution (conference)

arXiv link (url) [BibTex]

arXiv link (url) [BibTex]


no image
CausalCite: A Causal Formulation of Paper Citations

Kumar, I., Jin, Z., Mokhtarian, E., Guo, S., Chen, Y., Kiyavash, N., Sachan, M., Schölkopf, B.

Findings of the Association for Computational Linguistics (ACL), pages: 8395-8410, (Editors: Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek), Association for Computational Linguistics, August 2024 (conference)

arXiv link (url) [BibTex]

arXiv link (url) [BibTex]


no image
On the Growth of Mistakes in Differentially Private Online Learning: A Lower Bound Perspective

Dmitriev, D., Szabó, K., Sanyal, A.

Proceedings of the 37th Annual Conference on Learning Theory (COLT), 247, pages: 1379-1398, Proceedings of Machine Learning Research, (Editors: Agrawal, Shipra and Roth, Aaron), PMLR, July 2024, (talk) (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Robustness of Nonlinear Representation Learning

Buchholz, S., Schölkopf, B.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 4785-4821, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Diffusion Tempering Improves Parameter Estimation with Probabilistic Integrators for ODEs

Beck, J., Bosch, N., Deistler, M., Kadhim, K. L., Macke, J. H., Hennig, P., Berens, P.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 3305-3326, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

arXiv link (url) [BibTex]

arXiv link (url) [BibTex]


no image
Simultaneous identification of models and parameters of scientific simulators

Schröder, C., Macke, J. H.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 43895-43927, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Position: Understanding LLMs Requires More Than Statistical Generalization

Reizinger, P., Ujváry, S., Mészáros, A., Kerekes, A., Brendel, W., Huszár, F.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 42365-42390, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

arXiv link (url) [BibTex]

arXiv link (url) [BibTex]


no image
Diffusive Gibbs Sampling

Chen*, W., Zhang*, M., Paige, B., Hernández-Lobato, J. M., Barber, D.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 7731-7747, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024, *equal contribution (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
What Makes Safety Fine-tuning Methods Safe? A Mechanistic Study

Jain, S., Lubana, E. S., Oksuz, K., Joy, T., Torr, P. H. S., Sanyal, A., Dokania, P. K.

ICML 2024 Workshop on Mechanistic Interpretability (Spotlight), July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Improving Neural Additive Models with Bayesian Principles

Bouchiat, K., Immer, A., Yèche, H., Rätsch, G., Fortuin, V.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 4416-4443, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Unveiling CLIP Dynamics: Linear Mode Connectivity and Generalization

Abdolahpourrostam, A., Sanyal, A., Moosavi-Dezfooli, S.

ICML 2024 Workshop on Foundation Models in the Wild, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
A Sparsity Principle for Partially Observable Causal Representation Learning

Xu, D., Yao, D., Lachapelle, S., Taslakian, P., von Kügelgen, J., Locatello, F., Magliacane, S.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 55389-55433, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
A Measure-Theoretic Axiomatisation of Causality and Kernel Regression

Park, J.

University of Tübingen, Germany, July 2024 (phdthesis)

[BibTex]

[BibTex]


no image
Geometry-Aware Instrumental Variable Regression

Kremer, H., Schölkopf, B.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 25560-25582, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Targeted Reduction of Causal Models

Kekić, A., Schölkopf, B., Besserve, M.

40th Conference on Uncertainty in Artificial Intelligence (UAI), July 2024 (conference) To be published

arXiv link (url) [BibTex]

arXiv link (url) [BibTex]


no image
Stitching Manifolds: Leveraging Interaction to Compose Object Representations into Scenes

Keurti, H., Schölkopf, B., Aceituno, P. V., Grewe, B.

ICML 2024 Workshop on Geometry-grounded Representation Learning and Generative Modeling (GRaM), July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Provable Privacy with Non-Private Pre-Processing

Hu, Y., Sanyal, A., Schölkopf, B.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 19402-19437, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Implicit meta-learning may lead language models to trust more reliable sources

Krasheninnikov, D., Krasheninnikov, E., Mlodozeniec, Bruno K., Maharaj, T., Krueger, D.

Proceedings of the 41st International Conference on Machine Learning, 235, pages: 25534-25559, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Products, Abstractions and Inclusions of Causal Spaces

Buchholz, S., Park, J., Schölkopf, B.

40th Conference on Uncertainty in Artificial Intelligence (UAI), July 2024 (conference) To be published

arXiv [BibTex]

arXiv [BibTex]


no image
Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?

Opedal, A., Stolfo, A., Shirakami, H., Jiao, Y., Cotterell, R., Schölkopf, B., Saparov, A., Sachan, M.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 38762-38778, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Detecting and Identifying Selection Structure in Sequential Data

Zheng, Y., Tang, Z., Qiu, Y., Schölkopf, B., Zhang, K.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 61498-61525, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
The Role of Learning Algorithms in Collective Action

Ben-Dov*, O., Fawkes*, J., Samadi, S., Sanyal, A.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 3443-3461, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024, *equal contribution (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
All-in-one simulation-based inference

Gloeckler, M., Deistler, M., Weilbach, C. D., Wood, F., Macke, J. H.

Proceedings of the 41st International Conference on Machine Learning (ICML), 235, pages: 15735-15766, Proceedings of Machine Learning Research, (Editors: Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix), PMLR, July 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Accuracy on the wrong line: On the pitfalls of noisy data for OOD generalisation

Sanyal, A., Hu, Y., Yu, Y., Ma, Y., Wang, Y., Schölkopf, B.

ICML 2024 Next Generation of AI Safety Workshop (Oral), July 2024 (conference)

arXiv PDF [BibTex]

arXiv PDF [BibTex]


no image
GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs

Gao, G., Liu, W., Chen, A., Geiger, A., Schölkopf, B.

The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024 (conference) Accepted

[BibTex]

[BibTex]


no image
Automatic Generation of Model and Data Cards: A Step Towards Responsible AI

Liu, J., Li, W., Jin, Z., Diab, M.

Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), Volume 1: Long Papers, pages: 1975-1997, (Editors: Duh, Kevin and Gomez, Helena and Bethard, Steven), Association for Computational Linguistics, June 2024 (conference)

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Analyzing the Role of Semantic Representations in the Era of Large Language Models

Jin*, Z., Chen*, Y., Gonzalez*, F., Liu, J., Zhang, J., Michael, J., Schölkopf, B., Biab, M.

Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Volume 1: Long Papers, pages: 3781-3798, (Editors: Duh, Kevin and Gomez, Helena and Bethard, Steven), Association for Computational Linguistics, June 2024, *equal contribution (conference)

arXiv link (url) DOI [BibTex]

arXiv link (url) DOI [BibTex]


no image
Advancing Normalising Flows to Model Boltzmann Distributions

Stimper, V.

University of Cambridge, UK, Cambridge, June 2024, (Cambridge-Tübingen-Fellowship-Program) (phdthesis)

[BibTex]

[BibTex]


no image
Out-of-Variable Generalization for Discriminative Models

Guo, S., Wildberger, J., Schölkopf, B.

The Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference)

arXiv [BibTex]

arXiv [BibTex]


no image
Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding

Pace, A., Yèche, H., Schölkopf, B., Rätsch, G., Tennenholtz, G.

The Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference)

arXiv [BibTex]

arXiv [BibTex]


no image
Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion

Meterez*, A., Joudaki*, A., Orabona, F., Immer, A., Rätsch, G., Daneshmand, H.

The Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (conference)

arXiv [BibTex]

arXiv [BibTex]


no image
Certified private data release for sparse Lipschitz functions

Donhauser, K., Lokna, J., Sanyal, A., Boedihardjo, M., Hönig, R., Yang, F.

Proceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS), 238, pages: 1396-1404, Proceedings of Machine Learning Research, (Editors: Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen), PMLR, May 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Can Large Language Models Infer Causation from Correlation?

Jin, Z., Liu, J., Lyu, Z., Poff, S., Sachan, M., Mihalcea, R., Diab*, M., Schölkopf*, B.

The Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal supervision (conference)

arXiv [BibTex]

arXiv [BibTex]


no image
The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks

Spieler, A., Rahaman, N., Martius, G., Schölkopf, B., Levina, A.

In The Twelfth International Conference on Learning Representations (ICLR), May 2024 (inproceedings)

arXiv [BibTex]

arXiv [BibTex]


no image
Targeted Reduction of Causal Models

Kekić, A., Schölkopf, B., Besserve, M.

ICLR 2024 Workshop on AI4DifferentialEquations In Science, May 2024 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Open X-Embodiment: Robotic Learning Datasets and RT-X Models

Open X-Embodiment Collaboration ( incl. Guist, S., Schneider, J., Schölkopf, B., Büchler, D. ).

IEEE International Conference on Robotics and Automation (ICRA), pages: 6892-6903, May 2024 (conference)

arXiv link (url) DOI [BibTex]


no image
Identifying Policy Gradient Subspaces

Schneider, J., Schumacher, P., Guist, S., Chen, L., Häufle, D., Schölkopf, B., Büchler, D.

The Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference)

arXiv [BibTex]

arXiv [BibTex]


no image
Some Intriguing Aspects about Lipschitz Continuity of Neural Networks

Khromov*, G., Singh*, S. P.

The Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (conference)

arXiv [BibTex]

arXiv [BibTex]


Ghost on the Shell: An Expressive Representation of General {3D} Shapes
Ghost on the Shell: An Expressive Representation of General 3D Shapes

Liu, Z., Feng, Y., Xiu, Y., Liu, W., Paull, L., Black, M. J., Schölkopf, B.

In Proceedings of the Twelfth International Conference on Learning Representations, The Twelfth International Conference on Learning Representations, May 2024 (inproceedings)

Abstract
The creation of photorealistic virtual worlds requires the accurate modeling of 3D surface geometry for a wide range of objects. For this, meshes are appealing since they 1) enable fast physics-based rendering with realistic material and lighting, 2) support physical simulation, and 3) are memory-efficient for modern graphics pipelines. Recent work on reconstructing and statistically modeling 3D shape, however, has critiqued meshes as being topologically inflexible. To capture a wide range of object shapes, any 3D representation must be able to model solid, watertight, shapes as well as thin, open, surfaces. Recent work has focused on the former, and methods for reconstructing open surfaces do not support fast reconstruction with material and lighting or unconditional generative modelling. Inspired by the observation that open surfaces can be seen as islands floating on watertight surfaces, we parameterize open surfaces by defining a manifold signed distance field on watertight templates. With this parameterization, we further develop a grid-based and differentiable representation that parameterizes both watertight and non-watertight meshes of arbitrary topology. Our new representation, called Ghost-on-the-Shell (G-Shell), enables two important applications: differentiable rasterization-based reconstruction from multiview images and generative modelling of non-watertight meshes. We empirically demonstrate that G-Shell achieves state-of-the-art performance on non-watertight mesh reconstruction and generation tasks, while also performing effectively for watertight meshes.

Home Code Video Project [BibTex]

Home Code Video Project [BibTex]


no image
Skill or Luck? Return Decomposition via Advantage Functions

Pan, H., Schölkopf, B.

The Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference)

[BibTex]

[BibTex]


no image
Transformer Fusion with Optimal Transport

Imfeld*, M., Graldi*, J., Giordano*, M., Hofmann, T., Anagnostidis, S., Singh, S. P.

The Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (conference)

arXiv [BibTex]

arXiv [BibTex]


Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization

Liu, W., Qiu, Z., Feng, Y., Xiu, Y., Xue, Y., Yu, L., Feng, H., Liu, Z., Heo, J., Peng, S., Wen, Y., Black, M. J., Weller, A., Schölkopf, B.

In Proceedings of the Twelfth International Conference on Learning Representations (ICLR), The Twelfth International Conference on Learning Representations, May 2024 (inproceedings)

Abstract
Large foundation models are becoming ubiquitous, but training them from scratch is prohibitively expensive. Thus, efficiently adapting these powerful models to downstream tasks is increasingly important. In this paper, we study a principled finetuning paradigm -- Orthogonal Finetuning (OFT) -- for downstream task adaptation. Despite demonstrating good generalizability, OFT still uses a fairly large number of trainable parameters due to the high dimensionality of orthogonal matrices. To address this, we start by examining OFT from an information transmission perspective, and then identify a few key desiderata that enable better parameter-efficiency. Inspired by how the Cooley-Tukey fast Fourier transform algorithm enables efficient information transmission, we propose an efficient orthogonal parameterization using butterfly structures. We apply this parameterization to OFT, creating a novel parameter-efficient finetuning method, called Orthogonal Butterfly (BOFT). By subsuming OFT as a special case, BOFT introduces a generalized orthogonal finetuning framework. Finally, we conduct an extensive empirical study of adapting large vision transformers, large language models, and text-to-image diffusion models to various downstream tasks in vision and language.

Home Code HuggingFace project link (url) [BibTex]

Home Code HuggingFace project link (url) [BibTex]


no image
Causal Modeling with Stationary Diffusions

Lorch, L., Krause*, A., Schölkopf*, B.

Proceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS), 238, pages: 1927-1935, Proceedings of Machine Learning Research, (Editors: Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen), PMLR, May 2024, *equal supervision (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Multi-View Causal Representation Learning with Partial Observability

Yao, D., Xu, D., Lachapelle, S., Magliacane, S., Taslakian, P., Martius, G., von Kügelgen, J., Locatello, F.

The Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference)

arXiv [BibTex]

arXiv [BibTex]