Efficient data movement between nodes in a data center is essential for optimal performance of distributed workloads. With advancements in computing interconnection and memory, new opportunities have emerged. We propose a novel inter-node architecture and protocol called Flexible Memory Units (FMU) that uses optically disaggregated memory. FMUs can be dynamically allocated to different nodes during runtime using optical switches. The primary objective of FMUs is to use the disaggregated memory as temporary buffers during inter-node communication. We have implemented Simplecomm, an open-source simulator, to evaluate real MPI benchmarks using FMU. Our evaluation demonstrates significant speedups of up to 5.18× in communication-bound applications and 1.22× on computing-intensive applications, compared to a 100 Gbps InfiniBand interconnect.