Jesse Roberts

Jesse Roberts

Native Tennesseean Researching AI, Language, and Robotics

Posts by Tags

Grant

Fulbight Application

10 minute read

Published:

These are the notes I took for my first Fulbright application. It was not successful. I applied for Fall 24 in Ireland as phd student expecting to graduate by the end of summer 24. However, I didnt’t hear about the opportunity until the deadline was only a month away. I was able to get a letter of invitation from the host school.

Idea

Survey of Foundation Model Pre-Training Tasks

less than 1 minute read

Published:

Foundation models require appropriate data, architectures, and pre-training tasks. The former two considerations are typical in long-standing machine learning research. However, the need for appropriate pre-training tasks is relatively new. While novel foundation models require appropriate tasks, model architects may find important inspiration in task innovations from successful and unsuccessful foundation model development efforts. While not all tasks are appropriate for every architecture or dataset, the goal of this survey is to provide a reference from which foundation model architects may draw task inspiration and intuition for the creation of new models.

Was the Mosaic Law Necessarily Unique?

1 minute read

Published:

The Mosaic law is the result of divine inspiration colliding with the reality of leadership. However, what would have happened if God had chosen another to lead the Israelites? Would the resultant written law have been identical?

Manuscript

Survey of Foundation Model Pre-Training Tasks

less than 1 minute read

Published:

Foundation models require appropriate data, architectures, and pre-training tasks. The former two considerations are typical in long-standing machine learning research. However, the need for appropriate pre-training tasks is relatively new. While novel foundation models require appropriate tasks, model architects may find important inspiration in task innovations from successful and unsuccessful foundation model development efforts. While not all tasks are appropriate for every architecture or dataset, the goal of this survey is to provide a reference from which foundation model architects may draw task inspiration and intuition for the creation of new models.

Meeting notes

Question

Are LLMs Impulsive?

less than 1 minute read

Published:

Humans can, at times, have issues of impulse control. That is, we may have an idea of some action which is ill-conceived and rather than appropriately rejecting the idea, we act upon it. Would we expect language models to have similar context driven impulses?

Is a Recursive Universal Function Approximator Necessarily Turing Complete?

less than 1 minute read

Published:

Universal function approximation is unrelated to the computational expressivity of a computing model. Due to this it is unclear if universal function approximation coupled with recursion is necessarily sufficient to guarantee that a model is Turing complete. However, I believe that this is the case. Further, I believe it is provable.

Does Rhetoric Transcend Language?

1 minute read

Published:

Is discourse or rhetoric language transcendant? That is, does rhetoric require language and therefore is dependent on and shaped by it?

Was the Mosaic Law Necessarily Unique?

1 minute read

Published:

The Mosaic law is the result of divine inspiration colliding with the reality of leadership. However, what would have happened if God had chosen another to lead the Israelites? Would the resultant written law have been identical?

Research philosophy

How to Research

1 minute read

Published:

New researchers often struggle with developing independent research. Some advisors prefer to let students wonder and thereby learn to succeed (painful) and others prefer to guide (creates dependency). I have no answers for this philosophical question. However, I do have advice for young researchers gathered along my own journey.

Theology

Was the Mosaic Law Necessarily Unique?

1 minute read

Published:

The Mosaic law is the result of divine inspiration colliding with the reality of leadership. However, what would have happened if God had chosen another to lead the Israelites? Would the resultant written law have been identical?